2026-02-25T14:14:41.785 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-25T14:14:41.800 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-25T14:14:41.880 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209 branch: tentacle description: rados/verify/{centos_latest ceph clusters/fixed-4 d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/{bluestore/{alloc$/{btree} base mem$/{normal-2} onode-segment$/{none} write$/{random/{compr$/{no$/{no}} random}}}} rados read-affinity/balance tasks/rados_api_tests validater/valgrind} email: null first_in_suite: false flavor: default job_id: '70209' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: tentacle ceph: conf: client: debug monc: 20 debug ms: 1 debug objecter: 20 debug rados: 20 rados replica read policy: balance global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr2: false ms inject socket failures: 5000 ms type: async osd heartbeat grace: 160 osd op thread timeout: 150 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osd crush smoke test: false mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 mon warn on pool no app: false paxos service trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore allocator: btree bluestore block size: 96636764160 bluestore fsck on mount: true bluestore onode segment size: 0 bluestore write v2 random: true bluestore zero block detection: true crush location: root=default host=host-$id debug bluefs: 1 debug bluestore: 1 debug monc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd class default list: '*' osd class load list: '*' osd client watch timeout: 240 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd fast shutdown: false osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd mclock skip benchmark: true osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - reached quota - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(SLOW_OPS\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(OBJECT_MISPLACED\) - slow request - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(SLOW_OPS\) - slow request - OSD bench result - OSD_DOWN mon_bind_msgr2: false sha1: c824a7fa29b067db9ce5a381f59b175918299919 stop-daemons-timeout: 1200 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: debuginfo: true flavor: default sha1: c824a7fa29b067db9ce5a381f59b175918299919 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: tentacle sha1: c824a7fa29b067db9ce5a381f59b175918299919 owner: scheduled_jayaprakash@soko04.front.sepia.ceph.com priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.4 - osd.8 - osd.12 - - mon.b - osd.1 - osd.5 - osd.9 - osd.13 - - mon.c - osd.2 - osd.6 - osd.10 - osd.14 - - mgr.x - osd.3 - osd.7 - osd.11 - osd.15 - client.0 seed: 6877 sha1: c824a7fa29b067db9ce5a381f59b175918299919 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: tentacle suite_path: /home/teuthworker/src/github.com_ceph_ceph_c824a7fa29b067db9ce5a381f59b175918299919/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: c824a7fa29b067db9ce5a381f59b175918299919 targets: trial090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMifiaRszno3QLj/XgaKwqRtI6u3Q4VHWvtTcCq3FpldrfTPZCTEUyTVwFsaU6JI4yCpRrhz9XyiQHgrp7k5slU= trial093.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHFDe+hnunFXNfvF1s2St2q5un8074tf3AIaIXYMmQE8pu621PsHgx7FlyoYd8dDx03CLtRVk8pQVPVU5Skf1kE= trial094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA0sFA/J19RVGYTjoHPJti6YO6GdIa4wFWFcZ/quoxS6xXNHCjy6FCTeUJhS3oYG0MXZiIgtxjV19EBBUxMWcBU= trial189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFsxAT6wDgq/uTugbvXaaIajCMmcpBdDzSZBBOBD9/ujaLxo+HIHmpMuFku+WbwjKfUHYGl2fGoGcQj9GySoufc= tasks: - install: null - ceph: null - workunit: clients: client.0: - rados/test.sh env: ALLOW_TIMEOUTS: '1' timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - '-- Don''t run this test on tiny clusters if py_len(yaml.roles) < 4 then reject() end ' teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-25_14:08:21 tube: trial user: jayaprakash verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699 2026-02-25T14:14:41.880 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_c824a7fa29b067db9ce5a381f59b175918299919/qa; will attempt to use it 2026-02-25T14:14:41.880 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_c824a7fa29b067db9ce5a381f59b175918299919/qa/tasks 2026-02-25T14:14:41.881 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-25T14:14:41.882 INFO:teuthology.task.internal:Checking packages... 2026-02-25T14:14:41.882 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c824a7fa29b067db9ce5a381f59b175918299919' 2026-02-25T14:14:41.882 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-25T14:14:41.882 INFO:teuthology.packaging:ref: None 2026-02-25T14:14:41.882 INFO:teuthology.packaging:tag: None 2026-02-25T14:14:41.882 INFO:teuthology.packaging:branch: tentacle 2026-02-25T14:14:41.882 INFO:teuthology.packaging:sha1: c824a7fa29b067db9ce5a381f59b175918299919 2026-02-25T14:14:41.882 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=tentacle 2026-02-25T14:14:42.048 INFO:teuthology.task.internal:Found packages for ceph version 20.2.0-676.gc824a7fa 2026-02-25T14:14:42.050 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-25T14:14:42.062 INFO:teuthology.task.internal:no buildpackages task found 2026-02-25T14:14:42.062 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-25T14:14:42.075 INFO:teuthology.task.internal:Saving configuration 2026-02-25T14:14:42.095 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-25T14:14:42.108 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-25T14:14:42.196 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial090.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-25 14:11:11.670238', 'locked_by': 'scheduled_jayaprakash@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMifiaRszno3QLj/XgaKwqRtI6u3Q4VHWvtTcCq3FpldrfTPZCTEUyTVwFsaU6JI4yCpRrhz9XyiQHgrp7k5slU='} 2026-02-25T14:14:42.254 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial093.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-25 14:11:11.671184', 'locked_by': 'scheduled_jayaprakash@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHFDe+hnunFXNfvF1s2St2q5un8074tf3AIaIXYMmQE8pu621PsHgx7FlyoYd8dDx03CLtRVk8pQVPVU5Skf1kE='} 2026-02-25T14:14:42.309 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial094.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-25 14:11:11.671517', 'locked_by': 'scheduled_jayaprakash@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA0sFA/J19RVGYTjoHPJti6YO6GdIa4wFWFcZ/quoxS6xXNHCjy6FCTeUJhS3oYG0MXZiIgtxjV19EBBUxMWcBU='} 2026-02-25T14:14:42.365 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial189.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-25 14:11:11.670784', 'locked_by': 'scheduled_jayaprakash@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFsxAT6wDgq/uTugbvXaaIajCMmcpBdDzSZBBOBD9/ujaLxo+HIHmpMuFku+WbwjKfUHYGl2fGoGcQj9GySoufc='} 2026-02-25T14:14:42.365 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-25T14:14:42.376 INFO:teuthology.task.internal:roles: ubuntu@trial090.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'] 2026-02-25T14:14:42.376 INFO:teuthology.task.internal:roles: ubuntu@trial093.front.sepia.ceph.com - ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'] 2026-02-25T14:14:42.376 INFO:teuthology.task.internal:roles: ubuntu@trial094.front.sepia.ceph.com - ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'] 2026-02-25T14:14:42.376 INFO:teuthology.task.internal:roles: ubuntu@trial189.front.sepia.ceph.com - ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0'] 2026-02-25T14:14:42.376 INFO:teuthology.run_tasks:Running task console_log... 2026-02-25T14:14:42.625 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x796f3f7e9e40>, signals=[15]) 2026-02-25T14:14:42.625 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-25T14:14:42.637 INFO:teuthology.task.internal:Opening connections... 2026-02-25T14:14:42.637 DEBUG:teuthology.task.internal:connecting to ubuntu@trial090.front.sepia.ceph.com 2026-02-25T14:14:42.637 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:14:42.698 DEBUG:teuthology.task.internal:connecting to ubuntu@trial093.front.sepia.ceph.com 2026-02-25T14:14:42.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:14:42.761 DEBUG:teuthology.task.internal:connecting to ubuntu@trial094.front.sepia.ceph.com 2026-02-25T14:14:42.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:14:42.821 DEBUG:teuthology.task.internal:connecting to ubuntu@trial189.front.sepia.ceph.com 2026-02-25T14:14:42.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:14:42.881 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-25T14:14:42.894 DEBUG:teuthology.orchestra.run.trial090:> uname -m 2026-02-25T14:14:42.909 INFO:teuthology.orchestra.run.trial090.stdout:x86_64 2026-02-25T14:14:42.909 DEBUG:teuthology.orchestra.run.trial090:> cat /etc/os-release 2026-02-25T14:14:42.964 INFO:teuthology.orchestra.run.trial090.stdout:NAME="CentOS Stream" 2026-02-25T14:14:42.964 INFO:teuthology.orchestra.run.trial090.stdout:VERSION="9" 2026-02-25T14:14:42.964 INFO:teuthology.orchestra.run.trial090.stdout:ID="centos" 2026-02-25T14:14:42.964 INFO:teuthology.orchestra.run.trial090.stdout:ID_LIKE="rhel fedora" 2026-02-25T14:14:42.964 INFO:teuthology.orchestra.run.trial090.stdout:VERSION_ID="9" 2026-02-25T14:14:42.964 INFO:teuthology.orchestra.run.trial090.stdout:PLATFORM_ID="platform:el9" 2026-02-25T14:14:42.964 INFO:teuthology.orchestra.run.trial090.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-25T14:14:42.964 INFO:teuthology.orchestra.run.trial090.stdout:ANSI_COLOR="0;31" 2026-02-25T14:14:42.965 INFO:teuthology.orchestra.run.trial090.stdout:LOGO="fedora-logo-icon" 2026-02-25T14:14:42.965 INFO:teuthology.orchestra.run.trial090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-25T14:14:42.965 INFO:teuthology.orchestra.run.trial090.stdout:HOME_URL="https://centos.org/" 2026-02-25T14:14:42.965 INFO:teuthology.orchestra.run.trial090.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-25T14:14:42.965 INFO:teuthology.orchestra.run.trial090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-25T14:14:42.965 INFO:teuthology.orchestra.run.trial090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-25T14:14:42.965 INFO:teuthology.lock.ops:Updating trial090.front.sepia.ceph.com on lock server 2026-02-25T14:14:43.039 DEBUG:teuthology.orchestra.run.trial093:> uname -m 2026-02-25T14:14:43.055 INFO:teuthology.orchestra.run.trial093.stdout:x86_64 2026-02-25T14:14:43.055 DEBUG:teuthology.orchestra.run.trial093:> cat /etc/os-release 2026-02-25T14:14:43.110 INFO:teuthology.orchestra.run.trial093.stdout:NAME="CentOS Stream" 2026-02-25T14:14:43.111 INFO:teuthology.orchestra.run.trial093.stdout:VERSION="9" 2026-02-25T14:14:43.111 INFO:teuthology.orchestra.run.trial093.stdout:ID="centos" 2026-02-25T14:14:43.111 INFO:teuthology.orchestra.run.trial093.stdout:ID_LIKE="rhel fedora" 2026-02-25T14:14:43.111 INFO:teuthology.orchestra.run.trial093.stdout:VERSION_ID="9" 2026-02-25T14:14:43.111 INFO:teuthology.orchestra.run.trial093.stdout:PLATFORM_ID="platform:el9" 2026-02-25T14:14:43.111 INFO:teuthology.orchestra.run.trial093.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-25T14:14:43.111 INFO:teuthology.orchestra.run.trial093.stdout:ANSI_COLOR="0;31" 2026-02-25T14:14:43.111 INFO:teuthology.orchestra.run.trial093.stdout:LOGO="fedora-logo-icon" 2026-02-25T14:14:43.111 INFO:teuthology.orchestra.run.trial093.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-25T14:14:43.111 INFO:teuthology.orchestra.run.trial093.stdout:HOME_URL="https://centos.org/" 2026-02-25T14:14:43.111 INFO:teuthology.orchestra.run.trial093.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-25T14:14:43.111 INFO:teuthology.orchestra.run.trial093.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-25T14:14:43.111 INFO:teuthology.orchestra.run.trial093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-25T14:14:43.111 INFO:teuthology.lock.ops:Updating trial093.front.sepia.ceph.com on lock server 2026-02-25T14:14:43.186 DEBUG:teuthology.orchestra.run.trial094:> uname -m 2026-02-25T14:14:43.201 INFO:teuthology.orchestra.run.trial094.stdout:x86_64 2026-02-25T14:14:43.202 DEBUG:teuthology.orchestra.run.trial094:> cat /etc/os-release 2026-02-25T14:14:43.257 INFO:teuthology.orchestra.run.trial094.stdout:NAME="CentOS Stream" 2026-02-25T14:14:43.258 INFO:teuthology.orchestra.run.trial094.stdout:VERSION="9" 2026-02-25T14:14:43.258 INFO:teuthology.orchestra.run.trial094.stdout:ID="centos" 2026-02-25T14:14:43.258 INFO:teuthology.orchestra.run.trial094.stdout:ID_LIKE="rhel fedora" 2026-02-25T14:14:43.258 INFO:teuthology.orchestra.run.trial094.stdout:VERSION_ID="9" 2026-02-25T14:14:43.258 INFO:teuthology.orchestra.run.trial094.stdout:PLATFORM_ID="platform:el9" 2026-02-25T14:14:43.258 INFO:teuthology.orchestra.run.trial094.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-25T14:14:43.258 INFO:teuthology.orchestra.run.trial094.stdout:ANSI_COLOR="0;31" 2026-02-25T14:14:43.258 INFO:teuthology.orchestra.run.trial094.stdout:LOGO="fedora-logo-icon" 2026-02-25T14:14:43.258 INFO:teuthology.orchestra.run.trial094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-25T14:14:43.258 INFO:teuthology.orchestra.run.trial094.stdout:HOME_URL="https://centos.org/" 2026-02-25T14:14:43.258 INFO:teuthology.orchestra.run.trial094.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-25T14:14:43.258 INFO:teuthology.orchestra.run.trial094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-25T14:14:43.258 INFO:teuthology.orchestra.run.trial094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-25T14:14:43.259 INFO:teuthology.lock.ops:Updating trial094.front.sepia.ceph.com on lock server 2026-02-25T14:14:43.338 DEBUG:teuthology.orchestra.run.trial189:> uname -m 2026-02-25T14:14:43.354 INFO:teuthology.orchestra.run.trial189.stdout:x86_64 2026-02-25T14:14:43.354 DEBUG:teuthology.orchestra.run.trial189:> cat /etc/os-release 2026-02-25T14:14:43.409 INFO:teuthology.orchestra.run.trial189.stdout:NAME="CentOS Stream" 2026-02-25T14:14:43.409 INFO:teuthology.orchestra.run.trial189.stdout:VERSION="9" 2026-02-25T14:14:43.409 INFO:teuthology.orchestra.run.trial189.stdout:ID="centos" 2026-02-25T14:14:43.409 INFO:teuthology.orchestra.run.trial189.stdout:ID_LIKE="rhel fedora" 2026-02-25T14:14:43.409 INFO:teuthology.orchestra.run.trial189.stdout:VERSION_ID="9" 2026-02-25T14:14:43.409 INFO:teuthology.orchestra.run.trial189.stdout:PLATFORM_ID="platform:el9" 2026-02-25T14:14:43.409 INFO:teuthology.orchestra.run.trial189.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-25T14:14:43.409 INFO:teuthology.orchestra.run.trial189.stdout:ANSI_COLOR="0;31" 2026-02-25T14:14:43.409 INFO:teuthology.orchestra.run.trial189.stdout:LOGO="fedora-logo-icon" 2026-02-25T14:14:43.409 INFO:teuthology.orchestra.run.trial189.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-25T14:14:43.410 INFO:teuthology.orchestra.run.trial189.stdout:HOME_URL="https://centos.org/" 2026-02-25T14:14:43.410 INFO:teuthology.orchestra.run.trial189.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-25T14:14:43.410 INFO:teuthology.orchestra.run.trial189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-25T14:14:43.410 INFO:teuthology.orchestra.run.trial189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-25T14:14:43.410 INFO:teuthology.lock.ops:Updating trial189.front.sepia.ceph.com on lock server 2026-02-25T14:14:43.488 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-25T14:14:43.502 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-25T14:14:43.513 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-25T14:14:43.513 DEBUG:teuthology.orchestra.run.trial090:> test '!' -e /home/ubuntu/cephtest 2026-02-25T14:14:43.515 DEBUG:teuthology.orchestra.run.trial093:> test '!' -e /home/ubuntu/cephtest 2026-02-25T14:14:43.517 DEBUG:teuthology.orchestra.run.trial094:> test '!' -e /home/ubuntu/cephtest 2026-02-25T14:14:43.518 DEBUG:teuthology.orchestra.run.trial189:> test '!' -e /home/ubuntu/cephtest 2026-02-25T14:14:43.533 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-25T14:14:43.544 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-25T14:14:43.545 DEBUG:teuthology.orchestra.run.trial090:> test -z $(ls -A /var/lib/ceph) 2026-02-25T14:14:43.571 DEBUG:teuthology.orchestra.run.trial093:> test -z $(ls -A /var/lib/ceph) 2026-02-25T14:14:43.573 DEBUG:teuthology.orchestra.run.trial094:> test -z $(ls -A /var/lib/ceph) 2026-02-25T14:14:43.575 DEBUG:teuthology.orchestra.run.trial189:> test -z $(ls -A /var/lib/ceph) 2026-02-25T14:14:43.592 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-25T14:14:43.878 INFO:teuthology.run_tasks:Running task kernel... 2026-02-25T14:14:43.899 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-25T14:14:43.899 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-25T14:14:43.899 DEBUG:teuthology.orchestra.run.trial090:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-25T14:14:43.900 DEBUG:teuthology.orchestra.run.trial093:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-25T14:14:43.900 DEBUG:teuthology.orchestra.run.trial094:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-25T14:14:43.900 DEBUG:teuthology.orchestra.run.trial189:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-25T14:14:43.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T14:14:43.914 DEBUG:teuthology.orchestra.run.trial094:> uname -r 2026-02-25T14:14:43.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T14:14:43.915 DEBUG:teuthology.orchestra.run.trial093:> uname -r 2026-02-25T14:14:43.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T14:14:43.915 DEBUG:teuthology.orchestra.run.trial090:> uname -r 2026-02-25T14:14:43.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T14:14:43.915 DEBUG:teuthology.orchestra.run.trial189:> uname -r 2026-02-25T14:14:43.970 INFO:teuthology.orchestra.run.trial094.stdout:5.14.0-681.el9.x86_64 2026-02-25T14:14:43.970 INFO:teuthology.task.kernel:Running kernel on trial094: 5.14.0-681.el9.x86_64 2026-02-25T14:14:43.970 DEBUG:teuthology.orchestra.run.trial094:> sudo yum install -y kernel 2026-02-25T14:14:43.970 INFO:teuthology.orchestra.run.trial093.stdout:5.14.0-681.el9.x86_64 2026-02-25T14:14:43.970 INFO:teuthology.orchestra.run.trial090.stdout:5.14.0-681.el9.x86_64 2026-02-25T14:14:43.971 INFO:teuthology.task.kernel:Running kernel on trial093: 5.14.0-681.el9.x86_64 2026-02-25T14:14:43.971 DEBUG:teuthology.orchestra.run.trial093:> sudo yum install -y kernel 2026-02-25T14:14:43.971 INFO:teuthology.task.kernel:Running kernel on trial090: 5.14.0-681.el9.x86_64 2026-02-25T14:14:43.971 DEBUG:teuthology.orchestra.run.trial090:> sudo yum install -y kernel 2026-02-25T14:14:43.971 INFO:teuthology.orchestra.run.trial189.stdout:5.14.0-681.el9.x86_64 2026-02-25T14:14:43.971 INFO:teuthology.task.kernel:Running kernel on trial189: 5.14.0-681.el9.x86_64 2026-02-25T14:14:43.971 DEBUG:teuthology.orchestra.run.trial189:> sudo yum install -y kernel 2026-02-25T14:14:45.506 INFO:teuthology.orchestra.run.trial093.stdout:CentOS Stream 9 - BaseOS 7.8 MB/s | 8.9 MB 00:01 2026-02-25T14:14:46.266 INFO:teuthology.orchestra.run.trial090.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.9 MB 00:01 2026-02-25T14:14:47.856 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.9 MB 00:03 2026-02-25T14:14:48.245 INFO:teuthology.orchestra.run.trial093.stdout:CentOS Stream 9 - AppStream 12 MB/s | 27 MB 00:02 2026-02-25T14:14:48.761 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - AppStream 74 MB/s | 27 MB 00:00 2026-02-25T14:14:52.194 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 8.0 MB 00:01 2026-02-25T14:14:52.458 INFO:teuthology.orchestra.run.trial093.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 8.0 MB 00:01 2026-02-25T14:14:52.540 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.9 MB 00:08 2026-02-25T14:14:53.130 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - Extras packages 109 kB/s | 20 kB 00:00 2026-02-25T14:14:53.496 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - AppStream 62 MB/s | 27 MB 00:00 2026-02-25T14:14:53.541 INFO:teuthology.orchestra.run.trial093.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 20 kB 00:00 2026-02-25T14:14:53.732 INFO:teuthology.orchestra.run.trial189.stdout:Extra Packages for Enterprise Linux 38 MB/s | 20 MB 00:00 2026-02-25T14:14:56.099 INFO:teuthology.orchestra.run.trial093.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 20 MB 00:02 2026-02-25T14:14:56.900 INFO:teuthology.orchestra.run.trial189.stdout:lab-extras 2.1 MB/s | 50 kB 00:00 2026-02-25T14:14:57.272 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 8.0 MB 00:01 2026-02-25T14:14:57.912 INFO:teuthology.orchestra.run.trial189.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-25T14:14:57.912 INFO:teuthology.orchestra.run.trial189.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-25T14:14:57.932 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:14:57.933 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:14:57.933 INFO:teuthology.orchestra.run.trial189.stdout: Package Architecture Version Repository Size 2026-02-25T14:14:57.933 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:14:57.933 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:14:57.933 INFO:teuthology.orchestra.run.trial189.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-25T14:14:57.933 INFO:teuthology.orchestra.run.trial189.stdout:Installing dependencies: 2026-02-25T14:14:57.934 INFO:teuthology.orchestra.run.trial189.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-25T14:14:57.934 INFO:teuthology.orchestra.run.trial189.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-25T14:14:57.934 INFO:teuthology.orchestra.run.trial189.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-25T14:14:57.934 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:14:57.934 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:14:57.934 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:14:57.934 INFO:teuthology.orchestra.run.trial189.stdout:Install 4 Packages 2026-02-25T14:14:57.934 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:14:57.934 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 89 M 2026-02-25T14:14:57.934 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 135 M 2026-02-25T14:14:57.934 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:14:58.066 INFO:teuthology.orchestra.run.trial189.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 12 MB/s | 901 kB 00:00 2026-02-25T14:14:58.111 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - Extras packages 189 kB/s | 20 kB 00:00 2026-02-25T14:14:58.171 INFO:teuthology.orchestra.run.trial189.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 96 MB/s | 17 MB 00:00 2026-02-25T14:14:58.273 INFO:teuthology.orchestra.run.trial189.stdout:(3/4): kernel-modules-core-5.14.0-686.el9.x86_6 150 MB/s | 31 MB 00:00 2026-02-25T14:14:58.442 INFO:teuthology.orchestra.run.trial189.stdout:(4/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 89 MB/s | 40 MB 00:00 2026-02-25T14:14:58.444 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:14:58.444 INFO:teuthology.orchestra.run.trial189.stdout:Total 174 MB/s | 89 MB 00:00 2026-02-25T14:14:58.633 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:14:58.712 INFO:teuthology.orchestra.run.trial094.stdout:Extra Packages for Enterprise Linux 38 MB/s | 20 MB 00:00 2026-02-25T14:14:58.837 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:14:58.837 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:14:59.220 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:14:59.220 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:14:59.275 INFO:teuthology.orchestra.run.trial093.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-25T14:15:00.282 INFO:teuthology.orchestra.run.trial093.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-25T14:15:00.282 INFO:teuthology.orchestra.run.trial093.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-25T14:15:00.302 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:15:00.303 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:15:00.303 INFO:teuthology.orchestra.run.trial093.stdout: Package Architecture Version Repository Size 2026-02-25T14:15:00.303 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:15:00.303 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:15:00.304 INFO:teuthology.orchestra.run.trial093.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-25T14:15:00.304 INFO:teuthology.orchestra.run.trial093.stdout:Installing dependencies: 2026-02-25T14:15:00.304 INFO:teuthology.orchestra.run.trial093.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-25T14:15:00.304 INFO:teuthology.orchestra.run.trial093.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-25T14:15:00.304 INFO:teuthology.orchestra.run.trial093.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-25T14:15:00.304 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:15:00.304 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:15:00.304 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:15:00.304 INFO:teuthology.orchestra.run.trial093.stdout:Install 4 Packages 2026-02-25T14:15:00.304 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:15:00.304 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 89 M 2026-02-25T14:15:00.305 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 135 M 2026-02-25T14:15:00.305 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:15:00.331 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:15:00.462 INFO:teuthology.orchestra.run.trial093.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 13 MB/s | 901 kB 00:00 2026-02-25T14:15:00.596 INFO:teuthology.orchestra.run.trial093.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 85 MB/s | 17 MB 00:00 2026-02-25T14:15:00.620 INFO:teuthology.orchestra.run.trial189.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-25T14:15:00.642 INFO:teuthology.orchestra.run.trial189.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T14:15:00.942 INFO:teuthology.orchestra.run.trial093.stdout:(3/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 73 MB/s | 40 MB 00:00 2026-02-25T14:15:01.044 INFO:teuthology.orchestra.run.trial093.stdout:(4/4): kernel-modules-core-5.14.0-686.el9.x86_6 53 MB/s | 31 MB 00:00 2026-02-25T14:15:01.044 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:15:01.044 INFO:teuthology.orchestra.run.trial093.stdout:Total 120 MB/s | 89 MB 00:00 2026-02-25T14:15:01.240 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:15:01.446 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:15:01.446 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:15:01.830 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:15:01.830 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:15:01.862 INFO:teuthology.orchestra.run.trial094.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-25T14:15:02.877 INFO:teuthology.orchestra.run.trial094.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-25T14:15:02.877 INFO:teuthology.orchestra.run.trial094.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-25T14:15:02.896 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:15:02.898 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:15:02.898 INFO:teuthology.orchestra.run.trial094.stdout: Package Architecture Version Repository Size 2026-02-25T14:15:02.898 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:15:02.898 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:15:02.898 INFO:teuthology.orchestra.run.trial094.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-25T14:15:02.898 INFO:teuthology.orchestra.run.trial094.stdout:Installing dependencies: 2026-02-25T14:15:02.898 INFO:teuthology.orchestra.run.trial094.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-25T14:15:02.898 INFO:teuthology.orchestra.run.trial094.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-25T14:15:02.898 INFO:teuthology.orchestra.run.trial094.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-25T14:15:02.899 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:15:02.899 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:15:02.899 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:15:02.899 INFO:teuthology.orchestra.run.trial094.stdout:Install 4 Packages 2026-02-25T14:15:02.899 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:15:02.899 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 89 M 2026-02-25T14:15:02.899 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 135 M 2026-02-25T14:15:02.899 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:15:02.941 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:15:03.220 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T14:15:03.229 INFO:teuthology.orchestra.run.trial093.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-25T14:15:03.251 INFO:teuthology.orchestra.run.trial093.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T14:15:03.253 INFO:teuthology.orchestra.run.trial189.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T14:15:03.263 INFO:teuthology.orchestra.run.trial094.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 14 MB/s | 901 kB 00:00 2026-02-25T14:15:03.401 INFO:teuthology.orchestra.run.trial094.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 85 MB/s | 17 MB 00:00 2026-02-25T14:15:03.607 INFO:teuthology.orchestra.run.trial094.stdout:(3/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 98 MB/s | 40 MB 00:00 2026-02-25T14:15:04.176 INFO:teuthology.orchestra.run.trial094.stdout:(4/4): kernel-modules-core-5.14.0-686.el9.x86_6 34 MB/s | 31 MB 00:00 2026-02-25T14:15:04.176 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:15:04.176 INFO:teuthology.orchestra.run.trial094.stdout:Total 70 MB/s | 89 MB 00:01 2026-02-25T14:15:04.371 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:15:04.572 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:15:04.573 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:15:04.727 INFO:teuthology.orchestra.run.trial090.stdout:CentOS Stream 9 - AppStream 1.5 MB/s | 27 MB 00:17 2026-02-25T14:15:04.959 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:15:04.959 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:15:05.838 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T14:15:05.868 INFO:teuthology.orchestra.run.trial093.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T14:15:06.067 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:15:06.356 INFO:teuthology.orchestra.run.trial094.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-25T14:15:06.363 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T14:15:06.370 INFO:teuthology.orchestra.run.trial189.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:06.378 INFO:teuthology.orchestra.run.trial094.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T14:15:08.269 INFO:teuthology.orchestra.run.trial090.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 8.0 MB 00:01 2026-02-25T14:15:08.904 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T14:15:08.911 INFO:teuthology.orchestra.run.trial093.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:08.953 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T14:15:08.986 INFO:teuthology.orchestra.run.trial094.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T14:15:09.223 INFO:teuthology.orchestra.run.trial090.stdout:CentOS Stream 9 - Extras packages 91 kB/s | 20 kB 00:00 2026-02-25T14:15:09.626 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:10.628 INFO:teuthology.orchestra.run.trial090.stdout:Extra Packages for Enterprise Linux 15 MB/s | 20 MB 00:01 2026-02-25T14:15:12.022 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:12.251 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T14:15:12.257 INFO:teuthology.orchestra.run.trial094.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:13.788 INFO:teuthology.orchestra.run.trial090.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-25T14:15:14.800 INFO:teuthology.orchestra.run.trial090.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-25T14:15:14.800 INFO:teuthology.orchestra.run.trial090.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-25T14:15:14.820 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:15:14.821 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:15:14.821 INFO:teuthology.orchestra.run.trial090.stdout: Package Architecture Version Repository Size 2026-02-25T14:15:14.821 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:15:14.821 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:15:14.822 INFO:teuthology.orchestra.run.trial090.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-25T14:15:14.822 INFO:teuthology.orchestra.run.trial090.stdout:Installing dependencies: 2026-02-25T14:15:14.822 INFO:teuthology.orchestra.run.trial090.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-25T14:15:14.822 INFO:teuthology.orchestra.run.trial090.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-25T14:15:14.822 INFO:teuthology.orchestra.run.trial090.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-25T14:15:14.822 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:15:14.822 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:15:14.822 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:15:14.822 INFO:teuthology.orchestra.run.trial090.stdout:Install 4 Packages 2026-02-25T14:15:14.822 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:15:14.822 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 89 M 2026-02-25T14:15:14.823 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 135 M 2026-02-25T14:15:14.823 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:15:15.059 INFO:teuthology.orchestra.run.trial090.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 5.0 MB/s | 901 kB 00:00 2026-02-25T14:15:15.164 INFO:teuthology.orchestra.run.trial090.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 61 MB/s | 17 MB 00:00 2026-02-25T14:15:15.302 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:15.399 INFO:teuthology.orchestra.run.trial090.stdout:(3/4): kernel-modules-core-5.14.0-686.el9.x86_6 91 MB/s | 31 MB 00:00 2026-02-25T14:15:15.455 INFO:teuthology.orchestra.run.trial090.stdout:(4/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 70 MB/s | 40 MB 00:00 2026-02-25T14:15:15.455 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:15:15.455 INFO:teuthology.orchestra.run.trial090.stdout:Total 140 MB/s | 89 MB 00:00 2026-02-25T14:15:15.648 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:15:15.851 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:15:15.851 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:15:16.235 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:15:16.235 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:15:17.345 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:15:17.634 INFO:teuthology.orchestra.run.trial090.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-25T14:15:17.656 INFO:teuthology.orchestra.run.trial090.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T14:15:20.233 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T14:15:20.267 INFO:teuthology.orchestra.run.trial090.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T14:15:23.306 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T14:15:23.312 INFO:teuthology.orchestra.run.trial090.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:26.360 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:27.117 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:27.139 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:27.325 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:27.325 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-25T14:15:27.325 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T14:15:27.325 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T14:15:27.382 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:27.382 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:15:27.382 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:15:27.382 INFO:teuthology.orchestra.run.trial189.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-25T14:15:27.382 INFO:teuthology.orchestra.run.trial189.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-25T14:15:27.382 INFO:teuthology.orchestra.run.trial189.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-25T14:15:27.382 INFO:teuthology.orchestra.run.trial189.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-25T14:15:27.382 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:15:27.382 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:15:27.442 DEBUG:teuthology.orchestra.run.trial189:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T14:15:27.471 INFO:teuthology.orchestra.run.trial189.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T14:15:27.472 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T14:15:27.472 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-25T14:15:27.472 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-25T14:15:27.472 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-25T14:15:27.473 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2026-02-25T14:15:27.473 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial189.front.sepia.ceph.com, path=None, version=distro) 2026-02-25T14:15:27.473 DEBUG:teuthology.orchestra.run.trial189:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T14:15:27.537 INFO:teuthology.orchestra.run.trial189.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T14:15:27.539 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T14:15:27.539 DEBUG:teuthology.orchestra.run.trial189:> sudo rpm -qi grub2-tools 2026-02-25T14:15:27.607 INFO:teuthology.orchestra.run.trial189.stdout:Name : grub2-tools 2026-02-25T14:15:27.607 INFO:teuthology.orchestra.run.trial189.stdout:Epoch : 1 2026-02-25T14:15:27.607 INFO:teuthology.orchestra.run.trial189.stdout:Version : 2.06 2026-02-25T14:15:27.607 INFO:teuthology.orchestra.run.trial189.stdout:Release : 120.el9 2026-02-25T14:15:27.607 INFO:teuthology.orchestra.run.trial189.stdout:Architecture: x86_64 2026-02-25T14:15:27.607 INFO:teuthology.orchestra.run.trial189.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-25T14:15:27.607 INFO:teuthology.orchestra.run.trial189.stdout:Group : Unspecified 2026-02-25T14:15:27.607 INFO:teuthology.orchestra.run.trial189.stdout:Size : 9108611 2026-02-25T14:15:27.607 INFO:teuthology.orchestra.run.trial189.stdout:License : GPLv3+ 2026-02-25T14:15:27.607 INFO:teuthology.orchestra.run.trial189.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-25T14:15:27.607 INFO:teuthology.orchestra.run.trial189.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-25T14:15:27.607 INFO:teuthology.orchestra.run.trial189.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-25T14:15:27.607 INFO:teuthology.orchestra.run.trial189.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-25T14:15:27.607 INFO:teuthology.orchestra.run.trial189.stdout:Packager : builder@centos.org 2026-02-25T14:15:27.607 INFO:teuthology.orchestra.run.trial189.stdout:Vendor : CentOS 2026-02-25T14:15:27.608 INFO:teuthology.orchestra.run.trial189.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-25T14:15:27.608 INFO:teuthology.orchestra.run.trial189.stdout:Summary : Support tools for GRUB. 2026-02-25T14:15:27.608 INFO:teuthology.orchestra.run.trial189.stdout:Description : 2026-02-25T14:15:27.608 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:15:27.608 INFO:teuthology.orchestra.run.trial189.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-25T14:15:27.608 INFO:teuthology.orchestra.run.trial189.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-25T14:15:27.608 INFO:teuthology.orchestra.run.trial189.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-25T14:15:27.608 INFO:teuthology.orchestra.run.trial189.stdout:hardware devices. 2026-02-25T14:15:27.608 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:15:27.608 INFO:teuthology.orchestra.run.trial189.stdout:This subpackage provides tools for support of all platforms. 2026-02-25T14:15:27.609 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-25T14:15:27.609 INFO:teuthology.task.kernel:Updating grub on trial189 to boot 5.14.0-686.el9.x86_64 2026-02-25T14:15:27.609 DEBUG:teuthology.orchestra.run.trial189:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-25T14:15:27.671 INFO:teuthology.orchestra.run.trial189.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-25T14:15:27.672 DEBUG:teuthology.orchestra.run.trial189:> sudo command -v grubby && echo yes || echo no 2026-02-25T14:15:27.738 INFO:teuthology.orchestra.run.trial189.stdout:/sbin/grubby 2026-02-25T14:15:27.739 INFO:teuthology.orchestra.run.trial189.stdout:yes 2026-02-25T14:15:27.739 DEBUG:teuthology.orchestra.run.trial189:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-25T14:15:27.803 INFO:teuthology.orchestra.run.trial189.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-25T14:15:27.803 DEBUG:teuthology.orchestra.run.trial189:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-25T14:15:27.879 DEBUG:teuthology.orchestra.run.trial189:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-25T14:15:28.087 INFO:teuthology.orchestra.run.trial189.stderr:Generating grub configuration file ... 2026-02-25T14:15:28.153 INFO:teuthology.orchestra.run.trial189.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-25T14:15:28.160 INFO:teuthology.orchestra.run.trial189.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-25T14:15:28.410 INFO:teuthology.orchestra.run.trial189.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-25T14:15:28.416 INFO:teuthology.orchestra.run.trial189.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-25T14:15:28.451 INFO:teuthology.orchestra.run.trial189.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-25T14:15:28.458 INFO:teuthology.orchestra.run.trial189.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-25T14:15:28.489 INFO:teuthology.orchestra.run.trial189.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-25T14:15:28.496 INFO:teuthology.orchestra.run.trial189.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-25T14:15:28.519 INFO:teuthology.orchestra.run.trial189.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-25T14:15:28.525 INFO:teuthology.orchestra.run.trial189.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-25T14:15:28.612 INFO:teuthology.orchestra.run.trial189.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-25T14:15:28.620 INFO:teuthology.orchestra.run.trial189.stderr:done 2026-02-25T14:15:28.622 DEBUG:teuthology.orchestra.run.trial189:> test -d /sys/firmware/efi 2026-02-25T14:15:28.638 DEBUG:teuthology.orchestra.run.trial189:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-25T14:15:28.705 INFO:teuthology.orchestra.run.trial189.stdout:/boot/efi/EFI/BOOT 2026-02-25T14:15:28.705 INFO:teuthology.orchestra.run.trial189.stdout:/boot/efi/EFI/centos 2026-02-25T14:15:28.707 DEBUG:teuthology.orchestra.run.trial189:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-25T14:15:28.774 INFO:teuthology.orchestra.run.trial189.stdout:no 2026-02-25T14:15:28.775 DEBUG:teuthology.orchestra.run.trial189:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-25T14:15:28.840 INFO:teuthology.orchestra.run.trial189.stdout:yes 2026-02-25T14:15:28.840 DEBUG:teuthology.orchestra.run.trial189:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-25T14:15:28.904 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial189 2026-02-25T14:15:28.904 DEBUG:teuthology.orchestra.run.trial189:> sudo bash -c ' 2026-02-25T14:15:28.904 DEBUG:teuthology.orchestra.run.trial189:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-25T14:15:28.904 DEBUG:teuthology.orchestra.run.trial189:> ' 2026-02-25T14:15:29.457 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:29.479 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:29.666 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:29.667 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-25T14:15:29.667 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T14:15:29.667 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T14:15:29.724 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:29.725 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:15:29.725 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:15:29.725 INFO:teuthology.orchestra.run.trial093.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-25T14:15:29.725 INFO:teuthology.orchestra.run.trial093.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-25T14:15:29.725 INFO:teuthology.orchestra.run.trial093.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-25T14:15:29.725 INFO:teuthology.orchestra.run.trial093.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-25T14:15:29.725 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:15:29.725 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:15:29.787 DEBUG:teuthology.orchestra.run.trial093:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T14:15:29.816 INFO:teuthology.orchestra.run.trial093.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T14:15:29.817 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T14:15:29.817 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-25T14:15:29.817 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-25T14:15:29.817 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-25T14:15:29.818 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2026-02-25T14:15:29.818 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial093.front.sepia.ceph.com, path=None, version=distro) 2026-02-25T14:15:29.818 DEBUG:teuthology.orchestra.run.trial093:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T14:15:29.883 INFO:teuthology.orchestra.run.trial093.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T14:15:29.883 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T14:15:29.883 DEBUG:teuthology.orchestra.run.trial093:> sudo rpm -qi grub2-tools 2026-02-25T14:15:29.952 INFO:teuthology.orchestra.run.trial093.stdout:Name : grub2-tools 2026-02-25T14:15:29.952 INFO:teuthology.orchestra.run.trial093.stdout:Epoch : 1 2026-02-25T14:15:29.952 INFO:teuthology.orchestra.run.trial093.stdout:Version : 2.06 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:Release : 120.el9 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:Architecture: x86_64 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:Group : Unspecified 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:Size : 9108611 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:License : GPLv3+ 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:Packager : builder@centos.org 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:Vendor : CentOS 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:Summary : Support tools for GRUB. 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:Description : 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout:hardware devices. 2026-02-25T14:15:29.953 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:15:29.954 INFO:teuthology.orchestra.run.trial093.stdout:This subpackage provides tools for support of all platforms. 2026-02-25T14:15:29.954 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-25T14:15:29.954 INFO:teuthology.task.kernel:Updating grub on trial093 to boot 5.14.0-686.el9.x86_64 2026-02-25T14:15:29.954 DEBUG:teuthology.orchestra.run.trial093:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-25T14:15:30.017 INFO:teuthology.orchestra.run.trial093.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-25T14:15:30.018 DEBUG:teuthology.orchestra.run.trial093:> sudo command -v grubby && echo yes || echo no 2026-02-25T14:15:30.082 INFO:teuthology.orchestra.run.trial093.stdout:/sbin/grubby 2026-02-25T14:15:30.083 INFO:teuthology.orchestra.run.trial093.stdout:yes 2026-02-25T14:15:30.083 DEBUG:teuthology.orchestra.run.trial093:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-25T14:15:30.149 INFO:teuthology.orchestra.run.trial093.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-25T14:15:30.150 DEBUG:teuthology.orchestra.run.trial093:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-25T14:15:30.223 DEBUG:teuthology.orchestra.run.trial093:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-25T14:15:30.430 INFO:teuthology.orchestra.run.trial093.stderr:Generating grub configuration file ... 2026-02-25T14:15:30.498 INFO:teuthology.orchestra.run.trial093.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-25T14:15:30.504 INFO:teuthology.orchestra.run.trial093.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-25T14:15:30.755 INFO:teuthology.orchestra.run.trial093.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-25T14:15:30.762 INFO:teuthology.orchestra.run.trial093.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-25T14:15:30.795 INFO:teuthology.orchestra.run.trial093.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-25T14:15:30.802 INFO:teuthology.orchestra.run.trial093.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-25T14:15:30.833 INFO:teuthology.orchestra.run.trial093.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-25T14:15:30.840 INFO:teuthology.orchestra.run.trial093.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-25T14:15:30.863 INFO:teuthology.orchestra.run.trial093.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-25T14:15:30.869 INFO:teuthology.orchestra.run.trial093.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-25T14:15:30.956 INFO:teuthology.orchestra.run.trial093.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-25T14:15:30.964 INFO:teuthology.orchestra.run.trial093.stderr:done 2026-02-25T14:15:30.966 DEBUG:teuthology.orchestra.run.trial093:> test -d /sys/firmware/efi 2026-02-25T14:15:30.982 DEBUG:teuthology.orchestra.run.trial093:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-25T14:15:31.049 INFO:teuthology.orchestra.run.trial093.stdout:/boot/efi/EFI/BOOT 2026-02-25T14:15:31.049 INFO:teuthology.orchestra.run.trial093.stdout:/boot/efi/EFI/centos 2026-02-25T14:15:31.050 DEBUG:teuthology.orchestra.run.trial093:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-25T14:15:31.117 INFO:teuthology.orchestra.run.trial093.stdout:no 2026-02-25T14:15:31.118 DEBUG:teuthology.orchestra.run.trial093:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-25T14:15:31.181 INFO:teuthology.orchestra.run.trial093.stdout:yes 2026-02-25T14:15:31.182 DEBUG:teuthology.orchestra.run.trial093:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-25T14:15:31.247 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial093 2026-02-25T14:15:31.247 DEBUG:teuthology.orchestra.run.trial093:> sudo bash -c ' 2026-02-25T14:15:31.247 DEBUG:teuthology.orchestra.run.trial093:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-25T14:15:31.247 DEBUG:teuthology.orchestra.run.trial093:> ' 2026-02-25T14:15:32.775 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:32.797 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:32.986 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:32.986 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-25T14:15:32.986 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T14:15:32.986 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T14:15:33.044 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:33.044 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:15:33.044 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:15:33.044 INFO:teuthology.orchestra.run.trial094.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-25T14:15:33.044 INFO:teuthology.orchestra.run.trial094.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-25T14:15:33.044 INFO:teuthology.orchestra.run.trial094.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-25T14:15:33.044 INFO:teuthology.orchestra.run.trial094.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-25T14:15:33.044 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:15:33.045 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:15:33.106 DEBUG:teuthology.orchestra.run.trial094:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T14:15:33.136 INFO:teuthology.orchestra.run.trial094.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T14:15:33.136 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T14:15:33.137 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-25T14:15:33.137 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-25T14:15:33.137 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-25T14:15:33.137 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2026-02-25T14:15:33.137 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial094.front.sepia.ceph.com, path=None, version=distro) 2026-02-25T14:15:33.137 DEBUG:teuthology.orchestra.run.trial094:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T14:15:33.200 INFO:teuthology.orchestra.run.trial094.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T14:15:33.201 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T14:15:33.201 DEBUG:teuthology.orchestra.run.trial094:> sudo rpm -qi grub2-tools 2026-02-25T14:15:33.270 INFO:teuthology.orchestra.run.trial094.stdout:Name : grub2-tools 2026-02-25T14:15:33.270 INFO:teuthology.orchestra.run.trial094.stdout:Epoch : 1 2026-02-25T14:15:33.270 INFO:teuthology.orchestra.run.trial094.stdout:Version : 2.06 2026-02-25T14:15:33.270 INFO:teuthology.orchestra.run.trial094.stdout:Release : 120.el9 2026-02-25T14:15:33.270 INFO:teuthology.orchestra.run.trial094.stdout:Architecture: x86_64 2026-02-25T14:15:33.270 INFO:teuthology.orchestra.run.trial094.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-25T14:15:33.271 INFO:teuthology.orchestra.run.trial094.stdout:Group : Unspecified 2026-02-25T14:15:33.271 INFO:teuthology.orchestra.run.trial094.stdout:Size : 9108611 2026-02-25T14:15:33.271 INFO:teuthology.orchestra.run.trial094.stdout:License : GPLv3+ 2026-02-25T14:15:33.271 INFO:teuthology.orchestra.run.trial094.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-25T14:15:33.271 INFO:teuthology.orchestra.run.trial094.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-25T14:15:33.271 INFO:teuthology.orchestra.run.trial094.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-25T14:15:33.271 INFO:teuthology.orchestra.run.trial094.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-25T14:15:33.271 INFO:teuthology.orchestra.run.trial094.stdout:Packager : builder@centos.org 2026-02-25T14:15:33.271 INFO:teuthology.orchestra.run.trial094.stdout:Vendor : CentOS 2026-02-25T14:15:33.271 INFO:teuthology.orchestra.run.trial094.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-25T14:15:33.271 INFO:teuthology.orchestra.run.trial094.stdout:Summary : Support tools for GRUB. 2026-02-25T14:15:33.272 INFO:teuthology.orchestra.run.trial094.stdout:Description : 2026-02-25T14:15:33.272 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:15:33.272 INFO:teuthology.orchestra.run.trial094.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-25T14:15:33.272 INFO:teuthology.orchestra.run.trial094.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-25T14:15:33.272 INFO:teuthology.orchestra.run.trial094.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-25T14:15:33.272 INFO:teuthology.orchestra.run.trial094.stdout:hardware devices. 2026-02-25T14:15:33.272 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:15:33.272 INFO:teuthology.orchestra.run.trial094.stdout:This subpackage provides tools for support of all platforms. 2026-02-25T14:15:33.274 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-25T14:15:33.274 INFO:teuthology.task.kernel:Updating grub on trial094 to boot 5.14.0-686.el9.x86_64 2026-02-25T14:15:33.274 DEBUG:teuthology.orchestra.run.trial094:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-25T14:15:33.337 INFO:teuthology.orchestra.run.trial094.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-25T14:15:33.339 DEBUG:teuthology.orchestra.run.trial094:> sudo command -v grubby && echo yes || echo no 2026-02-25T14:15:33.404 INFO:teuthology.orchestra.run.trial094.stdout:/sbin/grubby 2026-02-25T14:15:33.405 INFO:teuthology.orchestra.run.trial094.stdout:yes 2026-02-25T14:15:33.405 DEBUG:teuthology.orchestra.run.trial094:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-25T14:15:33.470 INFO:teuthology.orchestra.run.trial094.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-25T14:15:33.470 DEBUG:teuthology.orchestra.run.trial094:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-25T14:15:33.543 DEBUG:teuthology.orchestra.run.trial094:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-25T14:15:33.749 INFO:teuthology.orchestra.run.trial094.stderr:Generating grub configuration file ... 2026-02-25T14:15:33.817 INFO:teuthology.orchestra.run.trial094.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-25T14:15:33.824 INFO:teuthology.orchestra.run.trial094.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-25T14:15:34.075 INFO:teuthology.orchestra.run.trial094.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-25T14:15:34.082 INFO:teuthology.orchestra.run.trial094.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-25T14:15:34.117 INFO:teuthology.orchestra.run.trial094.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-25T14:15:34.124 INFO:teuthology.orchestra.run.trial094.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-25T14:15:34.155 INFO:teuthology.orchestra.run.trial094.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-25T14:15:34.161 INFO:teuthology.orchestra.run.trial094.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-25T14:15:34.184 INFO:teuthology.orchestra.run.trial094.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-25T14:15:34.190 INFO:teuthology.orchestra.run.trial094.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-25T14:15:34.276 INFO:teuthology.orchestra.run.trial094.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-25T14:15:34.284 INFO:teuthology.orchestra.run.trial094.stderr:done 2026-02-25T14:15:34.286 DEBUG:teuthology.orchestra.run.trial094:> test -d /sys/firmware/efi 2026-02-25T14:15:34.303 DEBUG:teuthology.orchestra.run.trial094:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-25T14:15:34.370 INFO:teuthology.orchestra.run.trial094.stdout:/boot/efi/EFI/BOOT 2026-02-25T14:15:34.370 INFO:teuthology.orchestra.run.trial094.stdout:/boot/efi/EFI/centos 2026-02-25T14:15:34.372 DEBUG:teuthology.orchestra.run.trial094:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-25T14:15:34.437 INFO:teuthology.orchestra.run.trial094.stdout:no 2026-02-25T14:15:34.437 DEBUG:teuthology.orchestra.run.trial094:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-25T14:15:34.502 INFO:teuthology.orchestra.run.trial094.stdout:yes 2026-02-25T14:15:34.502 DEBUG:teuthology.orchestra.run.trial094:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-25T14:15:34.566 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial094 2026-02-25T14:15:34.566 DEBUG:teuthology.orchestra.run.trial094:> sudo bash -c ' 2026-02-25T14:15:34.566 DEBUG:teuthology.orchestra.run.trial094:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-25T14:15:34.566 DEBUG:teuthology.orchestra.run.trial094:> ' 2026-02-25T14:15:44.072 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:44.095 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:44.285 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:44.286 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-25T14:15:44.286 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T14:15:44.286 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T14:15:44.344 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T14:15:44.344 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:15:44.344 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:15:44.344 INFO:teuthology.orchestra.run.trial090.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-25T14:15:44.344 INFO:teuthology.orchestra.run.trial090.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-25T14:15:44.345 INFO:teuthology.orchestra.run.trial090.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-25T14:15:44.345 INFO:teuthology.orchestra.run.trial090.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-25T14:15:44.345 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:15:44.345 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:15:44.408 DEBUG:teuthology.orchestra.run.trial090:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T14:15:44.438 INFO:teuthology.orchestra.run.trial090.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T14:15:44.439 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T14:15:44.439 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-25T14:15:44.439 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-25T14:15:44.439 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-25T14:15:44.439 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-25T14:15:44.440 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial090.front.sepia.ceph.com, path=None, version=distro) 2026-02-25T14:15:44.440 DEBUG:teuthology.orchestra.run.trial090:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T14:15:44.506 INFO:teuthology.orchestra.run.trial090.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T14:15:44.507 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T14:15:44.508 DEBUG:teuthology.orchestra.run.trial090:> sudo rpm -qi grub2-tools 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Name : grub2-tools 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Epoch : 1 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Version : 2.06 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Release : 120.el9 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Architecture: x86_64 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Group : Unspecified 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Size : 9108611 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:License : GPLv3+ 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Packager : builder@centos.org 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Vendor : CentOS 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Summary : Support tools for GRUB. 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:Description : 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:hardware devices. 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:15:44.577 INFO:teuthology.orchestra.run.trial090.stdout:This subpackage provides tools for support of all platforms. 2026-02-25T14:15:44.579 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-25T14:15:44.580 INFO:teuthology.task.kernel:Updating grub on trial090 to boot 5.14.0-686.el9.x86_64 2026-02-25T14:15:44.580 DEBUG:teuthology.orchestra.run.trial090:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-25T14:15:44.650 INFO:teuthology.orchestra.run.trial090.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-25T14:15:44.651 DEBUG:teuthology.orchestra.run.trial090:> sudo command -v grubby && echo yes || echo no 2026-02-25T14:15:44.715 INFO:teuthology.orchestra.run.trial090.stdout:/sbin/grubby 2026-02-25T14:15:44.717 INFO:teuthology.orchestra.run.trial090.stdout:yes 2026-02-25T14:15:44.718 DEBUG:teuthology.orchestra.run.trial090:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-25T14:15:44.780 INFO:teuthology.orchestra.run.trial090.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-25T14:15:44.780 DEBUG:teuthology.orchestra.run.trial090:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-25T14:15:44.853 DEBUG:teuthology.orchestra.run.trial090:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-25T14:15:45.062 INFO:teuthology.orchestra.run.trial090.stderr:Generating grub configuration file ... 2026-02-25T14:15:45.131 INFO:teuthology.orchestra.run.trial090.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-25T14:15:45.138 INFO:teuthology.orchestra.run.trial090.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-25T14:15:45.392 INFO:teuthology.orchestra.run.trial090.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-25T14:15:45.399 INFO:teuthology.orchestra.run.trial090.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-25T14:15:45.433 INFO:teuthology.orchestra.run.trial090.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-25T14:15:45.439 INFO:teuthology.orchestra.run.trial090.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-25T14:15:45.471 INFO:teuthology.orchestra.run.trial090.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-25T14:15:45.477 INFO:teuthology.orchestra.run.trial090.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-25T14:15:45.500 INFO:teuthology.orchestra.run.trial090.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-25T14:15:45.507 INFO:teuthology.orchestra.run.trial090.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-25T14:15:45.594 INFO:teuthology.orchestra.run.trial090.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-25T14:15:45.602 INFO:teuthology.orchestra.run.trial090.stderr:done 2026-02-25T14:15:45.604 DEBUG:teuthology.orchestra.run.trial090:> test -d /sys/firmware/efi 2026-02-25T14:15:45.619 DEBUG:teuthology.orchestra.run.trial090:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-25T14:15:45.685 INFO:teuthology.orchestra.run.trial090.stdout:/boot/efi/EFI/BOOT 2026-02-25T14:15:45.686 INFO:teuthology.orchestra.run.trial090.stdout:/boot/efi/EFI/centos 2026-02-25T14:15:45.688 DEBUG:teuthology.orchestra.run.trial090:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-25T14:15:45.755 INFO:teuthology.orchestra.run.trial090.stdout:no 2026-02-25T14:15:45.756 DEBUG:teuthology.orchestra.run.trial090:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-25T14:15:45.820 INFO:teuthology.orchestra.run.trial090.stdout:yes 2026-02-25T14:15:45.821 DEBUG:teuthology.orchestra.run.trial090:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-25T14:15:45.886 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial090 2026-02-25T14:15:45.887 DEBUG:teuthology.orchestra.run.trial090:> sudo bash -c ' 2026-02-25T14:15:45.887 DEBUG:teuthology.orchestra.run.trial090:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-25T14:15:45.887 DEBUG:teuthology.orchestra.run.trial090:> ' 2026-02-25T14:15:58.947 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2026-02-25T14:15:58.948 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial189.front.sepia.ceph.com' 2026-02-25T14:15:58.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:16:01.289 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2026-02-25T14:16:01.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial093.front.sepia.ceph.com' 2026-02-25T14:16:01.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:16:04.609 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2026-02-25T14:16:04.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial094.front.sepia.ceph.com' 2026-02-25T14:16:04.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:16:15.929 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-25T14:16:15.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial090.front.sepia.ceph.com' 2026-02-25T14:16:15.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:16:58.993 DEBUG:teuthology.orchestra.remote:timed out 2026-02-25T14:17:01.294 DEBUG:teuthology.orchestra.remote:timed out 2026-02-25T14:17:04.615 DEBUG:teuthology.orchestra.remote:timed out 2026-02-25T14:17:07.997 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial189.front.sepia.ceph.com' 2026-02-25T14:17:07.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:17:08.088 DEBUG:teuthology.orchestra.run.trial189:> true 2026-02-25T14:17:08.251 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial189.front.sepia.ceph.com' 2026-02-25T14:17:08.251 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-686.el9.x86_64"... 2026-02-25T14:17:08.251 DEBUG:teuthology.orchestra.run.trial189:> uname -r 2026-02-25T14:17:08.307 INFO:teuthology.orchestra.run.trial189.stdout:5.14.0-686.el9.x86_64 2026-02-25T14:17:08.308 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-25T14:17:08.308 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-25T14:17:08.308 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-25T14:17:09.309 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2026-02-25T14:17:09.310 DEBUG:teuthology.orchestra.run.trial189:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-25T14:17:09.339 INFO:teuthology.orchestra.run.trial189.stdout:ttyS1 2026-02-25T14:17:09.365 DEBUG:teuthology.parallel:result is None 2026-02-25T14:17:10.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial093.front.sepia.ceph.com' 2026-02-25T14:17:10.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:17:10.415 DEBUG:teuthology.orchestra.run.trial093:> true 2026-02-25T14:17:10.579 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial093.front.sepia.ceph.com' 2026-02-25T14:17:10.580 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-686.el9.x86_64"... 2026-02-25T14:17:10.580 DEBUG:teuthology.orchestra.run.trial093:> uname -r 2026-02-25T14:17:10.635 INFO:teuthology.orchestra.run.trial093.stdout:5.14.0-686.el9.x86_64 2026-02-25T14:17:10.635 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-25T14:17:10.635 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-25T14:17:10.635 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-25T14:17:11.635 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2026-02-25T14:17:11.636 DEBUG:teuthology.orchestra.run.trial093:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-25T14:17:11.666 INFO:teuthology.orchestra.run.trial093.stdout:ttyS1 2026-02-25T14:17:11.691 DEBUG:teuthology.parallel:result is None 2026-02-25T14:17:13.616 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial094.front.sepia.ceph.com' 2026-02-25T14:17:13.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:17:13.714 DEBUG:teuthology.orchestra.run.trial094:> true 2026-02-25T14:17:13.874 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial094.front.sepia.ceph.com' 2026-02-25T14:17:13.874 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-686.el9.x86_64"... 2026-02-25T14:17:13.875 DEBUG:teuthology.orchestra.run.trial094:> uname -r 2026-02-25T14:17:13.930 INFO:teuthology.orchestra.run.trial094.stdout:5.14.0-686.el9.x86_64 2026-02-25T14:17:13.931 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-25T14:17:13.931 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-25T14:17:13.931 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-25T14:17:14.932 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2026-02-25T14:17:14.932 DEBUG:teuthology.orchestra.run.trial094:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-25T14:17:14.962 INFO:teuthology.orchestra.run.trial094.stdout:ttyS1 2026-02-25T14:17:14.986 DEBUG:teuthology.parallel:result is None 2026-02-25T14:17:15.933 DEBUG:teuthology.orchestra.remote:timed out 2026-02-25T14:17:24.934 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial090.front.sepia.ceph.com' 2026-02-25T14:17:24.936 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:17:25.020 DEBUG:teuthology.orchestra.run.trial090:> true 2026-02-25T14:17:25.175 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial090.front.sepia.ceph.com' 2026-02-25T14:17:25.176 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-686.el9.x86_64"... 2026-02-25T14:17:25.176 DEBUG:teuthology.orchestra.run.trial090:> uname -r 2026-02-25T14:17:25.233 INFO:teuthology.orchestra.run.trial090.stdout:5.14.0-686.el9.x86_64 2026-02-25T14:17:25.233 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-25T14:17:25.233 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-25T14:17:25.233 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-25T14:17:26.235 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-25T14:17:26.235 DEBUG:teuthology.orchestra.run.trial090:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-25T14:17:26.264 INFO:teuthology.orchestra.run.trial090.stdout:ttyS1 2026-02-25T14:17:26.287 DEBUG:teuthology.parallel:result is None 2026-02-25T14:17:26.288 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-25T14:17:26.300 INFO:teuthology.task.internal:Creating test directory... 2026-02-25T14:17:26.301 DEBUG:teuthology.orchestra.run.trial090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-25T14:17:26.330 DEBUG:teuthology.orchestra.run.trial093:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-25T14:17:26.333 DEBUG:teuthology.orchestra.run.trial094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-25T14:17:26.335 DEBUG:teuthology.orchestra.run.trial189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-25T14:17:26.354 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-25T14:17:26.365 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-25T14:17:26.379 INFO:teuthology.task.internal:Creating archive directory... 2026-02-25T14:17:26.380 DEBUG:teuthology.orchestra.run.trial090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-25T14:17:26.386 DEBUG:teuthology.orchestra.run.trial093:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-25T14:17:26.389 DEBUG:teuthology.orchestra.run.trial094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-25T14:17:26.391 DEBUG:teuthology.orchestra.run.trial189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-25T14:17:26.415 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-25T14:17:26.425 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-25T14:17:26.426 DEBUG:teuthology.orchestra.run.trial090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-25T14:17:26.443 DEBUG:teuthology.orchestra.run.trial093:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-25T14:17:26.446 DEBUG:teuthology.orchestra.run.trial094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-25T14:17:26.451 DEBUG:teuthology.orchestra.run.trial189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-25T14:17:26.465 INFO:teuthology.orchestra.run.trial090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T14:17:26.469 INFO:teuthology.orchestra.run.trial093.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T14:17:26.473 INFO:teuthology.orchestra.run.trial094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T14:17:26.474 INFO:teuthology.orchestra.run.trial090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T14:17:26.477 INFO:teuthology.orchestra.run.trial189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T14:17:26.478 INFO:teuthology.orchestra.run.trial093.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T14:17:26.482 INFO:teuthology.orchestra.run.trial094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T14:17:26.486 INFO:teuthology.orchestra.run.trial189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T14:17:26.488 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-25T14:17:26.506 INFO:teuthology.task.internal:Configuring sudo... 2026-02-25T14:17:26.507 DEBUG:teuthology.orchestra.run.trial090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-25T14:17:26.529 DEBUG:teuthology.orchestra.run.trial093:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-25T14:17:26.536 DEBUG:teuthology.orchestra.run.trial094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-25T14:17:26.560 DEBUG:teuthology.orchestra.run.trial189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-25T14:17:26.590 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-25T14:17:26.606 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-25T14:17:26.606 DEBUG:teuthology.orchestra.run.trial090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-25T14:17:26.611 DEBUG:teuthology.orchestra.run.trial093:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-25T14:17:26.612 DEBUG:teuthology.orchestra.run.trial094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-25T14:17:26.640 DEBUG:teuthology.orchestra.run.trial189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-25T14:17:26.657 DEBUG:teuthology.orchestra.run.trial090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T14:17:26.754 DEBUG:teuthology.orchestra.run.trial090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T14:17:26.779 DEBUG:teuthology.orchestra.run.trial090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T14:17:26.896 DEBUG:teuthology.orchestra.run.trial090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T14:17:26.920 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T14:17:26.920 DEBUG:teuthology.orchestra.run.trial090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-25T14:17:26.983 DEBUG:teuthology.orchestra.run.trial093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T14:17:27.064 DEBUG:teuthology.orchestra.run.trial093:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T14:17:27.089 DEBUG:teuthology.orchestra.run.trial093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T14:17:27.226 DEBUG:teuthology.orchestra.run.trial093:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T14:17:27.251 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T14:17:27.251 DEBUG:teuthology.orchestra.run.trial093:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-25T14:17:27.315 DEBUG:teuthology.orchestra.run.trial094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T14:17:27.392 DEBUG:teuthology.orchestra.run.trial094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T14:17:27.416 DEBUG:teuthology.orchestra.run.trial094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T14:17:27.534 DEBUG:teuthology.orchestra.run.trial094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T14:17:27.558 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T14:17:27.558 DEBUG:teuthology.orchestra.run.trial094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-25T14:17:27.621 DEBUG:teuthology.orchestra.run.trial189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T14:17:27.750 DEBUG:teuthology.orchestra.run.trial189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T14:17:27.775 DEBUG:teuthology.orchestra.run.trial189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T14:17:27.889 DEBUG:teuthology.orchestra.run.trial189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T14:17:27.912 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T14:17:27.912 DEBUG:teuthology.orchestra.run.trial189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-25T14:17:27.975 DEBUG:teuthology.orchestra.run.trial090:> sudo service rsyslog restart 2026-02-25T14:17:27.978 DEBUG:teuthology.orchestra.run.trial093:> sudo service rsyslog restart 2026-02-25T14:17:27.981 DEBUG:teuthology.orchestra.run.trial094:> sudo service rsyslog restart 2026-02-25T14:17:27.983 DEBUG:teuthology.orchestra.run.trial189:> sudo service rsyslog restart 2026-02-25T14:17:28.005 INFO:teuthology.orchestra.run.trial090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T14:17:28.007 INFO:teuthology.orchestra.run.trial093.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T14:17:28.010 INFO:teuthology.orchestra.run.trial094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T14:17:28.044 INFO:teuthology.orchestra.run.trial189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T14:17:28.446 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-25T14:17:28.458 INFO:teuthology.task.internal:Starting timer... 2026-02-25T14:17:28.459 INFO:teuthology.run_tasks:Running task pcp... 2026-02-25T14:17:28.482 INFO:teuthology.run_tasks:Running task selinux... 2026-02-25T14:17:28.499 DEBUG:teuthology.orchestra.run.trial090:> sudo service auditd rotate 2026-02-25T14:17:28.528 INFO:teuthology.orchestra.run.trial090.stdout:Rotating logs: 2026-02-25T14:17:28.529 DEBUG:teuthology.orchestra.run.trial093:> sudo service auditd rotate 2026-02-25T14:17:28.559 INFO:teuthology.orchestra.run.trial093.stdout:Rotating logs: 2026-02-25T14:17:28.560 DEBUG:teuthology.orchestra.run.trial094:> sudo service auditd rotate 2026-02-25T14:17:28.590 INFO:teuthology.orchestra.run.trial094.stdout:Rotating logs: 2026-02-25T14:17:28.591 DEBUG:teuthology.orchestra.run.trial189:> sudo service auditd rotate 2026-02-25T14:17:28.621 INFO:teuthology.orchestra.run.trial189.stdout:Rotating logs: 2026-02-25T14:17:28.623 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-25T14:17:28.623 DEBUG:teuthology.orchestra.run.trial090:> /usr/sbin/getenforce 2026-02-25T14:17:28.640 INFO:teuthology.orchestra.run.trial090.stdout:Permissive 2026-02-25T14:17:28.641 DEBUG:teuthology.orchestra.run.trial093:> /usr/sbin/getenforce 2026-02-25T14:17:28.658 INFO:teuthology.orchestra.run.trial093.stdout:Permissive 2026-02-25T14:17:28.659 DEBUG:teuthology.orchestra.run.trial094:> /usr/sbin/getenforce 2026-02-25T14:17:28.677 INFO:teuthology.orchestra.run.trial094.stdout:Permissive 2026-02-25T14:17:28.677 DEBUG:teuthology.orchestra.run.trial189:> /usr/sbin/getenforce 2026-02-25T14:17:28.694 INFO:teuthology.orchestra.run.trial189.stdout:Permissive 2026-02-25T14:17:28.694 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial090.front.sepia.ceph.com': 'permissive', 'ubuntu@trial093.front.sepia.ceph.com': 'permissive', 'ubuntu@trial094.front.sepia.ceph.com': 'permissive', 'ubuntu@trial189.front.sepia.ceph.com': 'permissive'} 2026-02-25T14:17:28.695 DEBUG:teuthology.orchestra.run.trial090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T14:17:28.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T14:17:28.720 DEBUG:teuthology.orchestra.run.trial093:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T14:17:28.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T14:17:28.745 DEBUG:teuthology.orchestra.run.trial094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T14:17:28.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T14:17:28.771 DEBUG:teuthology.orchestra.run.trial189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T14:17:28.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T14:17:28.798 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-25T14:17:28.798 DEBUG:teuthology.orchestra.run.trial090:> sudo /usr/sbin/setenforce permissive 2026-02-25T14:17:28.824 DEBUG:teuthology.orchestra.run.trial093:> sudo /usr/sbin/setenforce permissive 2026-02-25T14:17:28.850 DEBUG:teuthology.orchestra.run.trial094:> sudo /usr/sbin/setenforce permissive 2026-02-25T14:17:28.876 DEBUG:teuthology.orchestra.run.trial189:> sudo /usr/sbin/setenforce permissive 2026-02-25T14:17:28.902 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-25T14:17:28.918 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-25T14:17:28.932 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-25T14:17:28.934 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial090.front.sepia.ceph.com,trial093.front.sepia.ceph.com,trial094.front.sepia.ceph.com,trial189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-25T14:20:15.989 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial090.front.sepia.ceph.com'), Remote(name='ubuntu@trial093.front.sepia.ceph.com'), Remote(name='ubuntu@trial094.front.sepia.ceph.com'), Remote(name='ubuntu@trial189.front.sepia.ceph.com')] 2026-02-25T14:20:15.990 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial090.front.sepia.ceph.com' 2026-02-25T14:20:15.990 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:20:16.094 DEBUG:teuthology.orchestra.run.trial090:> true 2026-02-25T14:20:16.173 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial090.front.sepia.ceph.com' 2026-02-25T14:20:16.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial093.front.sepia.ceph.com' 2026-02-25T14:20:16.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:20:16.261 DEBUG:teuthology.orchestra.run.trial093:> true 2026-02-25T14:20:16.335 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial093.front.sepia.ceph.com' 2026-02-25T14:20:16.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial094.front.sepia.ceph.com' 2026-02-25T14:20:16.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:20:16.402 DEBUG:teuthology.orchestra.run.trial094:> true 2026-02-25T14:20:16.479 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial094.front.sepia.ceph.com' 2026-02-25T14:20:16.479 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial189.front.sepia.ceph.com' 2026-02-25T14:20:16.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T14:20:16.573 DEBUG:teuthology.orchestra.run.trial189:> true 2026-02-25T14:20:16.651 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial189.front.sepia.ceph.com' 2026-02-25T14:20:16.652 INFO:teuthology.run_tasks:Running task clock... 2026-02-25T14:20:16.664 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-25T14:20:16.665 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-25T14:20:16.665 DEBUG:teuthology.orchestra.run.trial090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T14:20:16.667 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-25T14:20:16.667 DEBUG:teuthology.orchestra.run.trial093:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T14:20:16.669 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-25T14:20:16.669 DEBUG:teuthology.orchestra.run.trial094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T14:20:16.671 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-25T14:20:16.671 DEBUG:teuthology.orchestra.run.trial189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T14:20:16.694 INFO:teuthology.orchestra.run.trial090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-25T14:20:16.694 INFO:teuthology.orchestra.run.trial093.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-25T14:20:16.697 INFO:teuthology.orchestra.run.trial094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-25T14:20:16.705 INFO:teuthology.orchestra.run.trial090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-25T14:20:16.706 INFO:teuthology.orchestra.run.trial093.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-25T14:20:16.709 INFO:teuthology.orchestra.run.trial094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-25T14:20:16.717 INFO:teuthology.orchestra.run.trial189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-25T14:20:16.724 INFO:teuthology.orchestra.run.trial090.stderr:sudo: ntpd: command not found 2026-02-25T14:20:16.726 INFO:teuthology.orchestra.run.trial093.stderr:sudo: ntpd: command not found 2026-02-25T14:20:16.728 INFO:teuthology.orchestra.run.trial094.stderr:sudo: ntpd: command not found 2026-02-25T14:20:16.729 INFO:teuthology.orchestra.run.trial189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-25T14:20:16.734 INFO:teuthology.orchestra.run.trial090.stdout:506 Cannot talk to daemon 2026-02-25T14:20:16.736 INFO:teuthology.orchestra.run.trial093.stdout:506 Cannot talk to daemon 2026-02-25T14:20:16.738 INFO:teuthology.orchestra.run.trial094.stdout:506 Cannot talk to daemon 2026-02-25T14:20:16.745 INFO:teuthology.orchestra.run.trial090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-25T14:20:16.747 INFO:teuthology.orchestra.run.trial093.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-25T14:20:16.749 INFO:teuthology.orchestra.run.trial189.stderr:sudo: ntpd: command not found 2026-02-25T14:20:16.749 INFO:teuthology.orchestra.run.trial094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-25T14:20:16.757 INFO:teuthology.orchestra.run.trial090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-25T14:20:16.758 INFO:teuthology.orchestra.run.trial093.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-25T14:20:16.759 INFO:teuthology.orchestra.run.trial189.stdout:506 Cannot talk to daemon 2026-02-25T14:20:16.760 INFO:teuthology.orchestra.run.trial094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-25T14:20:16.770 INFO:teuthology.orchestra.run.trial189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-25T14:20:16.782 INFO:teuthology.orchestra.run.trial189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-25T14:20:16.801 INFO:teuthology.orchestra.run.trial090.stderr:bash: line 1: ntpq: command not found 2026-02-25T14:20:16.802 INFO:teuthology.orchestra.run.trial090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T14:20:16.802 INFO:teuthology.orchestra.run.trial090.stdout:=============================================================================== 2026-02-25T14:20:16.806 INFO:teuthology.orchestra.run.trial093.stderr:bash: line 1: ntpq: command not found 2026-02-25T14:20:16.809 INFO:teuthology.orchestra.run.trial094.stderr:bash: line 1: ntpq: command not found 2026-02-25T14:20:16.827 INFO:teuthology.orchestra.run.trial093.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T14:20:16.828 INFO:teuthology.orchestra.run.trial093.stdout:=============================================================================== 2026-02-25T14:20:16.828 INFO:teuthology.orchestra.run.trial093.stdout:^? 172-233-189-68.ip.linode> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T14:20:16.828 INFO:teuthology.orchestra.run.trial093.stdout:^? 44.190.5.123 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T14:20:16.828 INFO:teuthology.orchestra.run.trial093.stdout:^? s2-b.time.mci1.us.rozint> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T14:20:16.828 INFO:teuthology.orchestra.run.trial093.stdout:^? 38.134.40.65 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T14:20:16.828 INFO:teuthology.orchestra.run.trial094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T14:20:16.828 INFO:teuthology.orchestra.run.trial094.stdout:=============================================================================== 2026-02-25T14:20:16.828 INFO:teuthology.orchestra.run.trial094.stdout:^? ip74-208-14-149.pbiaas.c> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T14:20:16.828 INFO:teuthology.orchestra.run.trial094.stdout:^? 44.190.5.123 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T14:20:16.828 INFO:teuthology.orchestra.run.trial094.stdout:^? s2-b.time.mci1.us.rozint> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T14:20:16.829 INFO:teuthology.orchestra.run.trial094.stdout:^? 38.134.40.65 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T14:20:16.836 INFO:teuthology.orchestra.run.trial189.stderr:bash: line 1: ntpq: command not found 2026-02-25T14:20:16.843 INFO:teuthology.orchestra.run.trial189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T14:20:16.843 INFO:teuthology.orchestra.run.trial189.stdout:=============================================================================== 2026-02-25T14:20:16.843 INFO:teuthology.orchestra.run.trial189.stdout:^? rn-02.koehn.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T14:20:16.844 INFO:teuthology.orchestra.run.trial189.stdout:^? 44.190.5.123 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T14:20:16.844 INFO:teuthology.orchestra.run.trial189.stdout:^? s2-b.time.mci1.us.rozint> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T14:20:16.844 INFO:teuthology.orchestra.run.trial189.stdout:^? 38.134.40.65 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T14:20:16.845 INFO:teuthology.run_tasks:Running task install... 2026-02-25T14:20:16.857 DEBUG:teuthology.task.install:project ceph 2026-02-25T14:20:16.857 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'c824a7fa29b067db9ce5a381f59b175918299919'}} 2026-02-25T14:20:16.857 DEBUG:teuthology.task.install:config {'debuginfo': True, 'flavor': 'default', 'sha1': 'c824a7fa29b067db9ce5a381f59b175918299919'} 2026-02-25T14:20:16.858 INFO:teuthology.task.install:Using flavor: default 2026-02-25T14:20:16.867 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2026-02-25T14:20:16.867 INFO:teuthology.task.install:extra packages: [] 2026-02-25T14:20:16.868 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c824a7fa29b067db9ce5a381f59b175918299919', 'tag': None, 'wait_for_package': False} 2026-02-25T14:20:16.868 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c824a7fa29b067db9ce5a381f59b175918299919 2026-02-25T14:20:16.870 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c824a7fa29b067db9ce5a381f59b175918299919', 'tag': None, 'wait_for_package': False} 2026-02-25T14:20:16.870 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c824a7fa29b067db9ce5a381f59b175918299919 2026-02-25T14:20:16.871 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c824a7fa29b067db9ce5a381f59b175918299919', 'tag': None, 'wait_for_package': False} 2026-02-25T14:20:16.871 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c824a7fa29b067db9ce5a381f59b175918299919 2026-02-25T14:20:16.871 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c824a7fa29b067db9ce5a381f59b175918299919', 'tag': None, 'wait_for_package': False} 2026-02-25T14:20:16.871 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c824a7fa29b067db9ce5a381f59b175918299919 2026-02-25T14:20:17.149 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/ 2026-02-25T14:20:17.149 INFO:teuthology.task.install.rpm:Package version is 20.2.0-676.gc824a7fa 2026-02-25T14:20:17.149 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/ 2026-02-25T14:20:17.149 INFO:teuthology.task.install.rpm:Package version is 20.2.0-676.gc824a7fa 2026-02-25T14:20:17.152 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/ 2026-02-25T14:20:17.152 INFO:teuthology.task.install.rpm:Package version is 20.2.0-676.gc824a7fa 2026-02-25T14:20:17.154 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/ 2026-02-25T14:20:17.154 INFO:teuthology.task.install.rpm:Package version is 20.2.0-676.gc824a7fa 2026-02-25T14:20:17.515 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T14:20:17.515 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T14:20:17.515 DEBUG:teuthology.orchestra.run.trial090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T14:20:17.518 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T14:20:17.518 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T14:20:17.518 DEBUG:teuthology.orchestra.run.trial094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T14:20:17.531 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T14:20:17.532 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T14:20:17.532 DEBUG:teuthology.orchestra.run.trial093:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T14:20:17.543 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-25T14:20:17.544 DEBUG:teuthology.orchestra.run.trial094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c824a7fa29b067db9ce5a381f59b175918299919/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T14:20:17.545 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-25T14:20:17.545 DEBUG:teuthology.orchestra.run.trial090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c824a7fa29b067db9ce5a381f59b175918299919/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T14:20:17.558 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-25T14:20:17.558 DEBUG:teuthology.orchestra.run.trial093:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c824a7fa29b067db9ce5a381f59b175918299919/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T14:20:17.578 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-ljs-perfissue/c824a7fa29b067db9ce5a381f59b175918299919/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T14:20:17.578 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T14:20:17.578 DEBUG:teuthology.orchestra.run.trial189:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T14:20:17.604 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-25T14:20:17.604 DEBUG:teuthology.orchestra.run.trial189:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c824a7fa29b067db9ce5a381f59b175918299919/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T14:20:17.607 DEBUG:teuthology.orchestra.run.trial094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T14:20:17.609 DEBUG:teuthology.orchestra.run.trial090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T14:20:17.621 DEBUG:teuthology.orchestra.run.trial093:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T14:20:17.668 DEBUG:teuthology.orchestra.run.trial189:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T14:20:17.680 DEBUG:teuthology.orchestra.run.trial094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T14:20:17.682 DEBUG:teuthology.orchestra.run.trial090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T14:20:17.694 DEBUG:teuthology.orchestra.run.trial093:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T14:20:17.742 DEBUG:teuthology.orchestra.run.trial189:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T14:20:17.744 INFO:teuthology.orchestra.run.trial094.stdout:check_obsoletes = 1 2026-02-25T14:20:17.745 DEBUG:teuthology.orchestra.run.trial094:> sudo yum clean all 2026-02-25T14:20:17.745 INFO:teuthology.orchestra.run.trial090.stdout:check_obsoletes = 1 2026-02-25T14:20:17.747 DEBUG:teuthology.orchestra.run.trial090:> sudo yum clean all 2026-02-25T14:20:17.757 INFO:teuthology.orchestra.run.trial093.stdout:check_obsoletes = 1 2026-02-25T14:20:17.758 DEBUG:teuthology.orchestra.run.trial093:> sudo yum clean all 2026-02-25T14:20:17.807 INFO:teuthology.orchestra.run.trial189.stdout:check_obsoletes = 1 2026-02-25T14:20:17.808 DEBUG:teuthology.orchestra.run.trial189:> sudo yum clean all 2026-02-25T14:20:17.891 INFO:teuthology.orchestra.run.trial094.stdout:41 files removed 2026-02-25T14:20:17.895 INFO:teuthology.orchestra.run.trial090.stdout:41 files removed 2026-02-25T14:20:17.905 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-radosgw 2026-02-25T14:20:17.908 INFO:teuthology.orchestra.run.trial093.stdout:41 files removed 2026-02-25T14:20:17.909 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-radosgw 2026-02-25T14:20:17.922 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-radosgw 2026-02-25T14:20:17.957 INFO:teuthology.orchestra.run.trial189.stdout:41 files removed 2026-02-25T14:20:17.971 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-radosgw 2026-02-25T14:20:18.647 INFO:teuthology.orchestra.run.trial094.stdout:ceph packages for x86_64 149 kB/s | 89 kB 00:00 2026-02-25T14:20:18.656 INFO:teuthology.orchestra.run.trial090.stdout:ceph packages for x86_64 149 kB/s | 89 kB 00:00 2026-02-25T14:20:18.668 INFO:teuthology.orchestra.run.trial093.stdout:ceph packages for x86_64 149 kB/s | 89 kB 00:00 2026-02-25T14:20:18.721 INFO:teuthology.orchestra.run.trial189.stdout:ceph packages for x86_64 147 kB/s | 89 kB 00:00 2026-02-25T14:20:19.251 INFO:teuthology.orchestra.run.trial094.stdout:ceph noarch packages 32 kB/s | 19 kB 00:00 2026-02-25T14:20:19.258 INFO:teuthology.orchestra.run.trial090.stdout:ceph noarch packages 32 kB/s | 19 kB 00:00 2026-02-25T14:20:19.278 INFO:teuthology.orchestra.run.trial093.stdout:ceph noarch packages 32 kB/s | 19 kB 00:00 2026-02-25T14:20:19.338 INFO:teuthology.orchestra.run.trial189.stdout:ceph noarch packages 32 kB/s | 19 kB 00:00 2026-02-25T14:20:19.805 INFO:teuthology.orchestra.run.trial094.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2026-02-25T14:20:19.812 INFO:teuthology.orchestra.run.trial090.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2026-02-25T14:20:19.826 INFO:teuthology.orchestra.run.trial093.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2026-02-25T14:20:19.885 INFO:teuthology.orchestra.run.trial189.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2026-02-25T14:20:20.715 INFO:teuthology.orchestra.run.trial093.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.9 MB 00:00 2026-02-25T14:20:20.929 INFO:teuthology.orchestra.run.trial090.stdout:CentOS Stream 9 - BaseOS 8.1 MB/s | 8.9 MB 00:01 2026-02-25T14:20:21.055 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - BaseOS 7.2 MB/s | 8.9 MB 00:01 2026-02-25T14:20:23.477 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.9 MB 00:03 2026-02-25T14:20:24.197 INFO:teuthology.orchestra.run.trial090.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 27 MB 00:02 2026-02-25T14:20:24.725 INFO:teuthology.orchestra.run.trial093.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 27 MB 00:03 2026-02-25T14:20:24.884 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 27 MB 00:03 2026-02-25T14:20:26.572 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-25T14:20:29.498 INFO:teuthology.orchestra.run.trial090.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 8.0 MB 00:02 2026-02-25T14:20:29.533 INFO:teuthology.orchestra.run.trial093.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 8.0 MB 00:02 2026-02-25T14:20:29.809 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 8.0 MB 00:02 2026-02-25T14:20:29.920 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 8.0 MB 00:00 2026-02-25T14:20:30.394 INFO:teuthology.orchestra.run.trial093.stdout:CentOS Stream 9 - Extras packages 187 kB/s | 20 kB 00:00 2026-02-25T14:20:30.707 INFO:teuthology.orchestra.run.trial090.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 20 kB 00:00 2026-02-25T14:20:30.716 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - Extras packages 122 kB/s | 20 kB 00:00 2026-02-25T14:20:30.774 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - Extras packages 192 kB/s | 20 kB 00:00 2026-02-25T14:20:31.030 INFO:teuthology.orchestra.run.trial093.stdout:Extra Packages for Enterprise Linux 36 MB/s | 20 MB 00:00 2026-02-25T14:20:31.043 INFO:teuthology.orchestra.run.trial090.stdout:Extra Packages for Enterprise Linux 74 MB/s | 20 MB 00:00 2026-02-25T14:20:31.549 INFO:teuthology.orchestra.run.trial189.stdout:Extra Packages for Enterprise Linux 29 MB/s | 20 MB 00:00 2026-02-25T14:20:31.723 INFO:teuthology.orchestra.run.trial094.stdout:Extra Packages for Enterprise Linux 22 MB/s | 20 MB 00:00 2026-02-25T14:20:34.229 INFO:teuthology.orchestra.run.trial090.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-25T14:20:34.244 INFO:teuthology.orchestra.run.trial093.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-25T14:20:34.716 INFO:teuthology.orchestra.run.trial189.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-25T14:20:34.952 INFO:teuthology.orchestra.run.trial094.stdout:lab-extras 2.4 MB/s | 50 kB 00:00 2026-02-25T14:20:35.448 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:20:35.450 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:20:35.450 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repository Size 2026-02-25T14:20:35.450 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:20:35.450 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:20:35.450 INFO:teuthology.orchestra.run.trial090.stdout: ceph-radosgw x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 24 M 2026-02-25T14:20:35.450 INFO:teuthology.orchestra.run.trial090.stdout:Upgrading: 2026-02-25T14:20:35.450 INFO:teuthology.orchestra.run.trial090.stdout: librados2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 3.5 M 2026-02-25T14:20:35.450 INFO:teuthology.orchestra.run.trial090.stdout: librbd1 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.8 M 2026-02-25T14:20:35.451 INFO:teuthology.orchestra.run.trial090.stdout:Installing dependencies: 2026-02-25T14:20:35.451 INFO:teuthology.orchestra.run.trial090.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-25T14:20:35.451 INFO:teuthology.orchestra.run.trial090.stdout: ceph-base x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 5.9 M 2026-02-25T14:20:35.451 INFO:teuthology.orchestra.run.trial090.stdout: ceph-common x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 24 M 2026-02-25T14:20:35.451 INFO:teuthology.orchestra.run.trial090.stdout: ceph-selinux x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 25 k 2026-02-25T14:20:35.451 INFO:teuthology.orchestra.run.trial090.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-25T14:20:35.451 INFO:teuthology.orchestra.run.trial090.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-25T14:20:35.451 INFO:teuthology.orchestra.run.trial090.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-25T14:20:35.451 INFO:teuthology.orchestra.run.trial090.stdout: libcephfs2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 869 k 2026-02-25T14:20:35.451 INFO:teuthology.orchestra.run.trial090.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-25T14:20:35.451 INFO:teuthology.orchestra.run.trial090.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-25T14:20:35.451 INFO:teuthology.orchestra.run.trial090.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-25T14:20:35.452 INFO:teuthology.orchestra.run.trial090.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-25T14:20:35.452 INFO:teuthology.orchestra.run.trial090.stdout: libradosstriper1 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 250 k 2026-02-25T14:20:35.452 INFO:teuthology.orchestra.run.trial090.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-25T14:20:35.452 INFO:teuthology.orchestra.run.trial090.stdout: librgw2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 6.4 M 2026-02-25T14:20:35.452 INFO:teuthology.orchestra.run.trial090.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-25T14:20:35.452 INFO:teuthology.orchestra.run.trial090.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-25T14:20:35.452 INFO:teuthology.orchestra.run.trial090.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-25T14:20:35.452 INFO:teuthology.orchestra.run.trial090.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-25T14:20:35.452 INFO:teuthology.orchestra.run.trial090.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-25T14:20:35.452 INFO:teuthology.orchestra.run.trial090.stdout: python3-ceph-argparse x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 45 k 2026-02-25T14:20:35.452 INFO:teuthology.orchestra.run.trial090.stdout: python3-ceph-common x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 175 k 2026-02-25T14:20:35.452 INFO:teuthology.orchestra.run.trial090.stdout: python3-cephfs x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 163 k 2026-02-25T14:20:35.452 INFO:teuthology.orchestra.run.trial090.stdout: python3-rados x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 324 k 2026-02-25T14:20:35.453 INFO:teuthology.orchestra.run.trial090.stdout: python3-rbd x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 305 k 2026-02-25T14:20:35.453 INFO:teuthology.orchestra.run.trial090.stdout: python3-rgw x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 99 k 2026-02-25T14:20:35.453 INFO:teuthology.orchestra.run.trial090.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-25T14:20:35.453 INFO:teuthology.orchestra.run.trial090.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-25T14:20:35.453 INFO:teuthology.orchestra.run.trial090.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-25T14:20:35.453 INFO:teuthology.orchestra.run.trial090.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-25T14:20:35.453 INFO:teuthology.orchestra.run.trial090.stdout:Installing weak dependencies: 2026-02-25T14:20:35.453 INFO:teuthology.orchestra.run.trial090.stdout: libcephfs-daemon x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 35 k 2026-02-25T14:20:35.453 INFO:teuthology.orchestra.run.trial090.stdout: libcephfs-proxy2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 24 k 2026-02-25T14:20:35.453 INFO:teuthology.orchestra.run.trial090.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-25T14:20:35.453 INFO:teuthology.orchestra.run.trial090.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-25T14:20:35.453 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:20:35.453 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:20:35.454 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:20:35.454 INFO:teuthology.orchestra.run.trial090.stdout:Install 35 Packages 2026-02-25T14:20:35.454 INFO:teuthology.orchestra.run.trial090.stdout:Upgrade 2 Packages 2026-02-25T14:20:35.454 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:20:35.455 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 79 M 2026-02-25T14:20:35.455 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:20:35.461 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:20:35.463 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:20:35.463 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repository Size 2026-02-25T14:20:35.463 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:20:35.463 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:20:35.464 INFO:teuthology.orchestra.run.trial093.stdout: ceph-radosgw x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 24 M 2026-02-25T14:20:35.464 INFO:teuthology.orchestra.run.trial093.stdout:Upgrading: 2026-02-25T14:20:35.464 INFO:teuthology.orchestra.run.trial093.stdout: librados2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 3.5 M 2026-02-25T14:20:35.464 INFO:teuthology.orchestra.run.trial093.stdout: librbd1 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.8 M 2026-02-25T14:20:35.464 INFO:teuthology.orchestra.run.trial093.stdout:Installing dependencies: 2026-02-25T14:20:35.464 INFO:teuthology.orchestra.run.trial093.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-25T14:20:35.464 INFO:teuthology.orchestra.run.trial093.stdout: ceph-base x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 5.9 M 2026-02-25T14:20:35.464 INFO:teuthology.orchestra.run.trial093.stdout: ceph-common x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 24 M 2026-02-25T14:20:35.464 INFO:teuthology.orchestra.run.trial093.stdout: ceph-selinux x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 25 k 2026-02-25T14:20:35.464 INFO:teuthology.orchestra.run.trial093.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-25T14:20:35.464 INFO:teuthology.orchestra.run.trial093.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-25T14:20:35.465 INFO:teuthology.orchestra.run.trial093.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-25T14:20:35.465 INFO:teuthology.orchestra.run.trial093.stdout: libcephfs2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 869 k 2026-02-25T14:20:35.465 INFO:teuthology.orchestra.run.trial093.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-25T14:20:35.465 INFO:teuthology.orchestra.run.trial093.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-25T14:20:35.465 INFO:teuthology.orchestra.run.trial093.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-25T14:20:35.465 INFO:teuthology.orchestra.run.trial093.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-25T14:20:35.465 INFO:teuthology.orchestra.run.trial093.stdout: libradosstriper1 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 250 k 2026-02-25T14:20:35.465 INFO:teuthology.orchestra.run.trial093.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-25T14:20:35.465 INFO:teuthology.orchestra.run.trial093.stdout: librgw2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 6.4 M 2026-02-25T14:20:35.465 INFO:teuthology.orchestra.run.trial093.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-25T14:20:35.465 INFO:teuthology.orchestra.run.trial093.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-25T14:20:35.465 INFO:teuthology.orchestra.run.trial093.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-25T14:20:35.465 INFO:teuthology.orchestra.run.trial093.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-25T14:20:35.465 INFO:teuthology.orchestra.run.trial093.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-25T14:20:35.466 INFO:teuthology.orchestra.run.trial093.stdout: python3-ceph-argparse x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 45 k 2026-02-25T14:20:35.466 INFO:teuthology.orchestra.run.trial093.stdout: python3-ceph-common x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 175 k 2026-02-25T14:20:35.466 INFO:teuthology.orchestra.run.trial093.stdout: python3-cephfs x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 163 k 2026-02-25T14:20:35.466 INFO:teuthology.orchestra.run.trial093.stdout: python3-rados x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 324 k 2026-02-25T14:20:35.466 INFO:teuthology.orchestra.run.trial093.stdout: python3-rbd x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 305 k 2026-02-25T14:20:35.466 INFO:teuthology.orchestra.run.trial093.stdout: python3-rgw x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 99 k 2026-02-25T14:20:35.466 INFO:teuthology.orchestra.run.trial093.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-25T14:20:35.466 INFO:teuthology.orchestra.run.trial093.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-25T14:20:35.466 INFO:teuthology.orchestra.run.trial093.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-25T14:20:35.466 INFO:teuthology.orchestra.run.trial093.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-25T14:20:35.466 INFO:teuthology.orchestra.run.trial093.stdout:Installing weak dependencies: 2026-02-25T14:20:35.466 INFO:teuthology.orchestra.run.trial093.stdout: libcephfs-daemon x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 35 k 2026-02-25T14:20:35.467 INFO:teuthology.orchestra.run.trial093.stdout: libcephfs-proxy2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 24 k 2026-02-25T14:20:35.467 INFO:teuthology.orchestra.run.trial093.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-25T14:20:35.467 INFO:teuthology.orchestra.run.trial093.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-25T14:20:35.467 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:20:35.467 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:20:35.467 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:20:35.467 INFO:teuthology.orchestra.run.trial093.stdout:Install 35 Packages 2026-02-25T14:20:35.467 INFO:teuthology.orchestra.run.trial093.stdout:Upgrade 2 Packages 2026-02-25T14:20:35.467 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:20:35.468 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 79 M 2026-02-25T14:20:35.468 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:20:35.936 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:20:35.938 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:20:35.938 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repository Size 2026-02-25T14:20:35.938 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:20:35.938 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:20:35.938 INFO:teuthology.orchestra.run.trial189.stdout: ceph-radosgw x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 24 M 2026-02-25T14:20:35.938 INFO:teuthology.orchestra.run.trial189.stdout:Upgrading: 2026-02-25T14:20:35.938 INFO:teuthology.orchestra.run.trial189.stdout: librados2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 3.5 M 2026-02-25T14:20:35.938 INFO:teuthology.orchestra.run.trial189.stdout: librbd1 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.8 M 2026-02-25T14:20:35.938 INFO:teuthology.orchestra.run.trial189.stdout:Installing dependencies: 2026-02-25T14:20:35.938 INFO:teuthology.orchestra.run.trial189.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-25T14:20:35.938 INFO:teuthology.orchestra.run.trial189.stdout: ceph-base x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 5.9 M 2026-02-25T14:20:35.938 INFO:teuthology.orchestra.run.trial189.stdout: ceph-common x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 24 M 2026-02-25T14:20:35.939 INFO:teuthology.orchestra.run.trial189.stdout: ceph-selinux x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 25 k 2026-02-25T14:20:35.939 INFO:teuthology.orchestra.run.trial189.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-25T14:20:35.939 INFO:teuthology.orchestra.run.trial189.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-25T14:20:35.939 INFO:teuthology.orchestra.run.trial189.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-25T14:20:35.939 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 869 k 2026-02-25T14:20:35.939 INFO:teuthology.orchestra.run.trial189.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-25T14:20:35.939 INFO:teuthology.orchestra.run.trial189.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-25T14:20:35.939 INFO:teuthology.orchestra.run.trial189.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-25T14:20:35.939 INFO:teuthology.orchestra.run.trial189.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-25T14:20:35.939 INFO:teuthology.orchestra.run.trial189.stdout: libradosstriper1 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 250 k 2026-02-25T14:20:35.939 INFO:teuthology.orchestra.run.trial189.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-25T14:20:35.939 INFO:teuthology.orchestra.run.trial189.stdout: librgw2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 6.4 M 2026-02-25T14:20:35.939 INFO:teuthology.orchestra.run.trial189.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-25T14:20:35.940 INFO:teuthology.orchestra.run.trial189.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-25T14:20:35.940 INFO:teuthology.orchestra.run.trial189.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-25T14:20:35.940 INFO:teuthology.orchestra.run.trial189.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-25T14:20:35.940 INFO:teuthology.orchestra.run.trial189.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-25T14:20:35.940 INFO:teuthology.orchestra.run.trial189.stdout: python3-ceph-argparse x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 45 k 2026-02-25T14:20:35.940 INFO:teuthology.orchestra.run.trial189.stdout: python3-ceph-common x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 175 k 2026-02-25T14:20:35.940 INFO:teuthology.orchestra.run.trial189.stdout: python3-cephfs x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 163 k 2026-02-25T14:20:35.940 INFO:teuthology.orchestra.run.trial189.stdout: python3-rados x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 324 k 2026-02-25T14:20:35.940 INFO:teuthology.orchestra.run.trial189.stdout: python3-rbd x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 305 k 2026-02-25T14:20:35.940 INFO:teuthology.orchestra.run.trial189.stdout: python3-rgw x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 99 k 2026-02-25T14:20:35.940 INFO:teuthology.orchestra.run.trial189.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-25T14:20:35.940 INFO:teuthology.orchestra.run.trial189.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-25T14:20:35.940 INFO:teuthology.orchestra.run.trial189.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-25T14:20:35.941 INFO:teuthology.orchestra.run.trial189.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-25T14:20:35.941 INFO:teuthology.orchestra.run.trial189.stdout:Installing weak dependencies: 2026-02-25T14:20:35.941 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs-daemon x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 35 k 2026-02-25T14:20:35.941 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs-proxy2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 24 k 2026-02-25T14:20:35.941 INFO:teuthology.orchestra.run.trial189.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-25T14:20:35.941 INFO:teuthology.orchestra.run.trial189.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-25T14:20:35.941 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:20:35.941 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:20:35.941 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:20:35.941 INFO:teuthology.orchestra.run.trial189.stdout:Install 35 Packages 2026-02-25T14:20:35.941 INFO:teuthology.orchestra.run.trial189.stdout:Upgrade 2 Packages 2026-02-25T14:20:35.942 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:20:35.942 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 79 M 2026-02-25T14:20:35.942 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:20:36.174 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:20:36.176 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:20:36.176 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repository Size 2026-02-25T14:20:36.176 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:20:36.176 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:20:36.176 INFO:teuthology.orchestra.run.trial094.stdout: ceph-radosgw x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 24 M 2026-02-25T14:20:36.176 INFO:teuthology.orchestra.run.trial094.stdout:Upgrading: 2026-02-25T14:20:36.176 INFO:teuthology.orchestra.run.trial094.stdout: librados2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 3.5 M 2026-02-25T14:20:36.177 INFO:teuthology.orchestra.run.trial094.stdout: librbd1 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.8 M 2026-02-25T14:20:36.177 INFO:teuthology.orchestra.run.trial094.stdout:Installing dependencies: 2026-02-25T14:20:36.177 INFO:teuthology.orchestra.run.trial094.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-25T14:20:36.177 INFO:teuthology.orchestra.run.trial094.stdout: ceph-base x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 5.9 M 2026-02-25T14:20:36.177 INFO:teuthology.orchestra.run.trial094.stdout: ceph-common x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 24 M 2026-02-25T14:20:36.177 INFO:teuthology.orchestra.run.trial094.stdout: ceph-selinux x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 25 k 2026-02-25T14:20:36.177 INFO:teuthology.orchestra.run.trial094.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-25T14:20:36.177 INFO:teuthology.orchestra.run.trial094.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-25T14:20:36.177 INFO:teuthology.orchestra.run.trial094.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-25T14:20:36.177 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 869 k 2026-02-25T14:20:36.177 INFO:teuthology.orchestra.run.trial094.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-25T14:20:36.177 INFO:teuthology.orchestra.run.trial094.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-25T14:20:36.178 INFO:teuthology.orchestra.run.trial094.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-25T14:20:36.178 INFO:teuthology.orchestra.run.trial094.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-25T14:20:36.178 INFO:teuthology.orchestra.run.trial094.stdout: libradosstriper1 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 250 k 2026-02-25T14:20:36.178 INFO:teuthology.orchestra.run.trial094.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-25T14:20:36.178 INFO:teuthology.orchestra.run.trial094.stdout: librgw2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 6.4 M 2026-02-25T14:20:36.178 INFO:teuthology.orchestra.run.trial094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-25T14:20:36.178 INFO:teuthology.orchestra.run.trial094.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-25T14:20:36.178 INFO:teuthology.orchestra.run.trial094.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-25T14:20:36.178 INFO:teuthology.orchestra.run.trial094.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-25T14:20:36.178 INFO:teuthology.orchestra.run.trial094.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-25T14:20:36.178 INFO:teuthology.orchestra.run.trial094.stdout: python3-ceph-argparse x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 45 k 2026-02-25T14:20:36.178 INFO:teuthology.orchestra.run.trial094.stdout: python3-ceph-common x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 175 k 2026-02-25T14:20:36.179 INFO:teuthology.orchestra.run.trial094.stdout: python3-cephfs x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 163 k 2026-02-25T14:20:36.179 INFO:teuthology.orchestra.run.trial094.stdout: python3-rados x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 324 k 2026-02-25T14:20:36.179 INFO:teuthology.orchestra.run.trial094.stdout: python3-rbd x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 305 k 2026-02-25T14:20:36.179 INFO:teuthology.orchestra.run.trial094.stdout: python3-rgw x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 99 k 2026-02-25T14:20:36.179 INFO:teuthology.orchestra.run.trial094.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-25T14:20:36.179 INFO:teuthology.orchestra.run.trial094.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-25T14:20:36.179 INFO:teuthology.orchestra.run.trial094.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-25T14:20:36.179 INFO:teuthology.orchestra.run.trial094.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-25T14:20:36.179 INFO:teuthology.orchestra.run.trial094.stdout:Installing weak dependencies: 2026-02-25T14:20:36.179 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs-daemon x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 35 k 2026-02-25T14:20:36.179 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs-proxy2 x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 24 k 2026-02-25T14:20:36.179 INFO:teuthology.orchestra.run.trial094.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-25T14:20:36.180 INFO:teuthology.orchestra.run.trial094.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-25T14:20:36.180 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:20:36.180 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:20:36.180 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:20:36.180 INFO:teuthology.orchestra.run.trial094.stdout:Install 35 Packages 2026-02-25T14:20:36.180 INFO:teuthology.orchestra.run.trial094.stdout:Upgrade 2 Packages 2026-02-25T14:20:36.180 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:20:36.181 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 79 M 2026-02-25T14:20:36.181 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:20:36.589 INFO:teuthology.orchestra.run.trial093.stdout:(1/37): ceph-base-20.2.0-676.gc824a7fa.el9.x86_ 6.8 MB/s | 5.9 MB 00:00 2026-02-25T14:20:36.640 INFO:teuthology.orchestra.run.trial093.stdout:(2/37): ceph-selinux-20.2.0-676.gc824a7fa.el9.x 494 kB/s | 25 kB 00:00 2026-02-25T14:20:36.678 INFO:teuthology.orchestra.run.trial093.stdout:(3/37): libcephfs-daemon-20.2.0-676.gc824a7fa.e 925 kB/s | 35 kB 00:00 2026-02-25T14:20:36.719 INFO:teuthology.orchestra.run.trial093.stdout:(4/37): libcephfs-proxy2-20.2.0-676.gc824a7fa.e 595 kB/s | 24 kB 00:00 2026-02-25T14:20:36.776 INFO:teuthology.orchestra.run.trial093.stdout:(5/37): libcephfs2-20.2.0-676.gc824a7fa.el9.x86 15 MB/s | 869 kB 00:00 2026-02-25T14:20:36.809 INFO:teuthology.orchestra.run.trial093.stdout:(6/37): libradosstriper1-20.2.0-676.gc824a7fa.e 7.5 MB/s | 250 kB 00:00 2026-02-25T14:20:37.242 INFO:teuthology.orchestra.run.trial093.stdout:(7/37): librgw2-20.2.0-676.gc824a7fa.el9.x86_64 15 MB/s | 6.4 MB 00:00 2026-02-25T14:20:37.245 INFO:teuthology.orchestra.run.trial094.stdout:(1/37): ceph-base-20.2.0-676.gc824a7fa.el9.x86_ 7.2 MB/s | 5.9 MB 00:00 2026-02-25T14:20:37.281 INFO:teuthology.orchestra.run.trial093.stdout:(8/37): python3-ceph-argparse-20.2.0-676.gc824a 1.1 MB/s | 45 kB 00:00 2026-02-25T14:20:37.282 INFO:teuthology.orchestra.run.trial094.stdout:(2/37): ceph-selinux-20.2.0-676.gc824a7fa.el9.x 686 kB/s | 25 kB 00:00 2026-02-25T14:20:37.324 INFO:teuthology.orchestra.run.trial093.stdout:(9/37): python3-ceph-common-20.2.0-676.gc824a7f 4.0 MB/s | 175 kB 00:00 2026-02-25T14:20:37.324 INFO:teuthology.orchestra.run.trial094.stdout:(3/37): libcephfs-daemon-20.2.0-676.gc824a7fa.e 842 kB/s | 35 kB 00:00 2026-02-25T14:20:37.367 INFO:teuthology.orchestra.run.trial093.stdout:(10/37): python3-cephfs-20.2.0-676.gc824a7fa.el 3.7 MB/s | 163 kB 00:00 2026-02-25T14:20:37.368 INFO:teuthology.orchestra.run.trial094.stdout:(4/37): libcephfs-proxy2-20.2.0-676.gc824a7fa.e 558 kB/s | 24 kB 00:00 2026-02-25T14:20:37.415 INFO:teuthology.orchestra.run.trial093.stdout:(11/37): python3-rados-20.2.0-676.gc824a7fa.el9 6.7 MB/s | 324 kB 00:00 2026-02-25T14:20:37.474 INFO:teuthology.orchestra.run.trial093.stdout:(12/37): python3-rbd-20.2.0-676.gc824a7fa.el9.x 5.0 MB/s | 305 kB 00:00 2026-02-25T14:20:37.477 INFO:teuthology.orchestra.run.trial094.stdout:(5/37): libcephfs2-20.2.0-676.gc824a7fa.el9.x86 7.8 MB/s | 869 kB 00:00 2026-02-25T14:20:37.520 INFO:teuthology.orchestra.run.trial093.stdout:(13/37): python3-rgw-20.2.0-676.gc824a7fa.el9.x 2.1 MB/s | 99 kB 00:00 2026-02-25T14:20:37.526 INFO:teuthology.orchestra.run.trial094.stdout:(6/37): libradosstriper1-20.2.0-676.gc824a7fa.e 5.0 MB/s | 250 kB 00:00 2026-02-25T14:20:38.071 INFO:teuthology.orchestra.run.trial094.stdout:(7/37): librgw2-20.2.0-676.gc824a7fa.el9.x86_64 12 MB/s | 6.4 MB 00:00 2026-02-25T14:20:38.077 INFO:teuthology.orchestra.run.trial093.stdout:(14/37): mailcap-2.1.49-5.el9.noarch.rpm 59 kB/s | 33 kB 00:00 2026-02-25T14:20:38.127 INFO:teuthology.orchestra.run.trial094.stdout:(8/37): python3-ceph-argparse-20.2.0-676.gc824a 812 kB/s | 45 kB 00:00 2026-02-25T14:20:38.186 INFO:teuthology.orchestra.run.trial094.stdout:(9/37): python3-ceph-common-20.2.0-676.gc824a7f 2.9 MB/s | 175 kB 00:00 2026-02-25T14:20:38.246 INFO:teuthology.orchestra.run.trial094.stdout:(10/37): python3-cephfs-20.2.0-676.gc824a7fa.el 2.6 MB/s | 163 kB 00:00 2026-02-25T14:20:38.311 INFO:teuthology.orchestra.run.trial094.stdout:(11/37): python3-rados-20.2.0-676.gc824a7fa.el9 4.9 MB/s | 324 kB 00:00 2026-02-25T14:20:38.377 INFO:teuthology.orchestra.run.trial094.stdout:(12/37): python3-rbd-20.2.0-676.gc824a7fa.el9.x 4.5 MB/s | 305 kB 00:00 2026-02-25T14:20:38.441 INFO:teuthology.orchestra.run.trial094.stdout:(13/37): python3-rgw-20.2.0-676.gc824a7fa.el9.x 1.5 MB/s | 99 kB 00:00 2026-02-25T14:20:38.580 INFO:teuthology.orchestra.run.trial094.stdout:(14/37): ceph-common-20.2.0-676.gc824a7fa.el9.x 11 MB/s | 24 MB 00:02 2026-02-25T14:20:38.714 INFO:teuthology.orchestra.run.trial094.stdout:(15/37): mailcap-2.1.49-5.el9.noarch.rpm 121 kB/s | 33 kB 00:00 2026-02-25T14:20:38.943 INFO:teuthology.orchestra.run.trial094.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 407 kB/s | 93 kB 00:00 2026-02-25T14:20:39.155 INFO:teuthology.orchestra.run.trial189.stdout:(1/37): ceph-base-20.2.0-676.gc824a7fa.el9.x86_ 2.4 MB/s | 5.9 MB 00:02 2026-02-25T14:20:39.224 INFO:teuthology.orchestra.run.trial189.stdout:(2/37): ceph-selinux-20.2.0-676.gc824a7fa.el9.x 367 kB/s | 25 kB 00:00 2026-02-25T14:20:39.256 INFO:teuthology.orchestra.run.trial094.stdout:(17/37): nvme-cli-2.16-1.el9.x86_64.rpm 1.7 MB/s | 1.2 MB 00:00 2026-02-25T14:20:39.292 INFO:teuthology.orchestra.run.trial189.stdout:(3/37): libcephfs-daemon-20.2.0-676.gc824a7fa.e 517 kB/s | 35 kB 00:00 2026-02-25T14:20:39.360 INFO:teuthology.orchestra.run.trial189.stdout:(4/37): libcephfs-proxy2-20.2.0-676.gc824a7fa.e 354 kB/s | 24 kB 00:00 2026-02-25T14:20:39.570 INFO:teuthology.orchestra.run.trial093.stdout:(15/37): nvme-cli-2.16-1.el9.x86_64.rpm 789 kB/s | 1.2 MB 00:01 2026-02-25T14:20:39.636 INFO:teuthology.orchestra.run.trial093.stdout:(16/37): ceph-radosgw-20.2.0-676.gc824a7fa.el9. 6.0 MB/s | 24 MB 00:03 2026-02-25T14:20:39.721 INFO:teuthology.orchestra.run.trial189.stdout:(5/37): libcephfs2-20.2.0-676.gc824a7fa.el9.x86 2.4 MB/s | 869 kB 00:00 2026-02-25T14:20:39.735 INFO:teuthology.orchestra.run.trial094.stdout:(18/37): boost-program-options-1.75.0-13.el9.x8 131 kB/s | 104 kB 00:00 2026-02-25T14:20:39.776 INFO:teuthology.orchestra.run.trial189.stdout:(6/37): libradosstriper1-20.2.0-676.gc824a7fa.e 4.5 MB/s | 250 kB 00:00 2026-02-25T14:20:39.806 INFO:teuthology.orchestra.run.trial093.stdout:(17/37): pciutils-3.7.0-7.el9.x86_64.rpm 394 kB/s | 93 kB 00:00 2026-02-25T14:20:39.890 INFO:teuthology.orchestra.run.trial094.stdout:(19/37): libnbd-1.20.3-4.el9.x86_64.rpm 258 kB/s | 164 kB 00:00 2026-02-25T14:20:40.049 INFO:teuthology.orchestra.run.trial094.stdout:(20/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 509 kB/s | 160 kB 00:00 2026-02-25T14:20:40.050 INFO:teuthology.orchestra.run.trial094.stdout:(21/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 284 kB/s | 45 kB 00:00 2026-02-25T14:20:40.122 INFO:teuthology.orchestra.run.trial093.stdout:(18/37): libnbd-1.20.3-4.el9.x86_64.rpm 520 kB/s | 164 kB 00:00 2026-02-25T14:20:40.352 INFO:teuthology.orchestra.run.trial093.stdout:(19/37): boost-program-options-1.75.0-13.el9.x8 145 kB/s | 104 kB 00:00 2026-02-25T14:20:40.364 INFO:teuthology.orchestra.run.trial094.stdout:(22/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 931 kB/s | 292 kB 00:00 2026-02-25T14:20:40.436 INFO:teuthology.orchestra.run.trial093.stdout:(20/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 509 kB/s | 160 kB 00:00 2026-02-25T14:20:40.509 INFO:teuthology.orchestra.run.trial093.stdout:(21/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 287 kB/s | 45 kB 00:00 2026-02-25T14:20:40.523 INFO:teuthology.orchestra.run.trial094.stdout:(23/37): qatlib-25.08.0-2.el9.x86_64.rpm 1.5 MB/s | 240 kB 00:00 2026-02-25T14:20:40.682 INFO:teuthology.orchestra.run.trial094.stdout:(24/37): qatlib-service-25.08.0-2.el9.x86_64.rp 232 kB/s | 37 kB 00:00 2026-02-25T14:20:40.841 INFO:teuthology.orchestra.run.trial094.stdout:(25/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 418 kB/s | 66 kB 00:00 2026-02-25T14:20:40.913 INFO:teuthology.orchestra.run.trial094.stdout:(26/37): librdkafka-1.6.1-102.el9.x86_64.rpm 767 kB/s | 662 kB 00:00 2026-02-25T14:20:40.982 INFO:teuthology.orchestra.run.trial093.stdout:(22/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 618 kB/s | 292 kB 00:00 2026-02-25T14:20:41.018 INFO:teuthology.orchestra.run.trial094.stdout:(27/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.7 MB/s | 308 kB 00:00 2026-02-25T14:20:41.041 INFO:teuthology.orchestra.run.trial094.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 1.0 MB/s | 25 kB 00:00 2026-02-25T14:20:41.065 INFO:teuthology.orchestra.run.trial094.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2026-02-25T14:20:41.090 INFO:teuthology.orchestra.run.trial094.stdout:(30/37): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2026-02-25T14:20:41.154 INFO:teuthology.orchestra.run.trial094.stdout:(31/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 13 MB/s | 838 kB 00:00 2026-02-25T14:20:41.171 INFO:teuthology.orchestra.run.trial189.stdout:(7/37): librgw2-20.2.0-676.gc824a7fa.el9.x86_64 4.6 MB/s | 6.4 MB 00:01 2026-02-25T14:20:41.174 INFO:teuthology.orchestra.run.trial094.stdout:(32/37): libarrow-9.0.0-15.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2026-02-25T14:20:41.179 INFO:teuthology.orchestra.run.trial094.stdout:(33/37): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2026-02-25T14:20:41.204 INFO:teuthology.orchestra.run.trial094.stdout:(34/37): thrift-0.15.0-4.el9.x86_64.rpm 53 MB/s | 1.6 MB 00:00 2026-02-25T14:20:41.229 INFO:teuthology.orchestra.run.trial189.stdout:(8/37): python3-ceph-argparse-20.2.0-676.gc824a 778 kB/s | 45 kB 00:00 2026-02-25T14:20:41.289 INFO:teuthology.orchestra.run.trial189.stdout:(9/37): python3-ceph-common-20.2.0-676.gc824a7f 2.8 MB/s | 175 kB 00:00 2026-02-25T14:20:41.301 INFO:teuthology.orchestra.run.trial093.stdout:(23/37): librdkafka-1.6.1-102.el9.x86_64.rpm 766 kB/s | 662 kB 00:00 2026-02-25T14:20:41.351 INFO:teuthology.orchestra.run.trial189.stdout:(10/37): python3-cephfs-20.2.0-676.gc824a7fa.el 2.6 MB/s | 163 kB 00:00 2026-02-25T14:20:41.375 INFO:teuthology.orchestra.run.trial093.stdout:(24/37): qatlib-25.08.0-2.el9.x86_64.rpm 609 kB/s | 240 kB 00:00 2026-02-25T14:20:41.459 INFO:teuthology.orchestra.run.trial093.stdout:(25/37): qatlib-service-25.08.0-2.el9.x86_64.rp 235 kB/s | 37 kB 00:00 2026-02-25T14:20:41.478 INFO:teuthology.orchestra.run.trial189.stdout:(11/37): python3-rados-20.2.0-676.gc824a7fa.el9 2.5 MB/s | 324 kB 00:00 2026-02-25T14:20:41.535 INFO:teuthology.orchestra.run.trial093.stdout:(26/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 416 kB/s | 66 kB 00:00 2026-02-25T14:20:41.693 INFO:teuthology.orchestra.run.trial093.stdout:(27/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.3 MB/s | 308 kB 00:00 2026-02-25T14:20:41.725 INFO:teuthology.orchestra.run.trial093.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 770 kB/s | 25 kB 00:00 2026-02-25T14:20:41.759 INFO:teuthology.orchestra.run.trial093.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2026-02-25T14:20:41.820 INFO:teuthology.orchestra.run.trial189.stdout:(12/37): python3-rbd-20.2.0-676.gc824a7fa.el9.x 892 kB/s | 305 kB 00:00 2026-02-25T14:20:41.906 INFO:teuthology.orchestra.run.trial094.stdout:(35/37): librbd1-20.2.0-676.gc824a7fa.el9.x86_6 4.1 MB/s | 2.8 MB 00:00 2026-02-25T14:20:41.939 INFO:teuthology.orchestra.run.trial094.stdout:(36/37): librados2-20.2.0-676.gc824a7fa.el9.x86 4.6 MB/s | 3.5 MB 00:00 2026-02-25T14:20:41.947 INFO:teuthology.orchestra.run.trial093.stdout:(30/37): libarrow-9.0.0-15.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2026-02-25T14:20:41.947 INFO:teuthology.orchestra.run.trial093.stdout:(31/37): libunwind-1.6.2-1.el9.x86_64.rpm 357 kB/s | 67 kB 00:00 2026-02-25T14:20:41.975 INFO:teuthology.orchestra.run.trial189.stdout:(13/37): python3-rgw-20.2.0-676.gc824a7fa.el9.x 646 kB/s | 99 kB 00:00 2026-02-25T14:20:42.037 INFO:teuthology.orchestra.run.trial093.stdout:(32/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 9.1 MB/s | 838 kB 00:00 2026-02-25T14:20:42.043 INFO:teuthology.orchestra.run.trial093.stdout:(33/37): re2-20211101-20.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2026-02-25T14:20:42.148 INFO:teuthology.orchestra.run.trial093.stdout:(34/37): thrift-0.15.0-4.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2026-02-25T14:20:42.279 INFO:teuthology.orchestra.run.trial090.stdout:(1/37): ceph-base-20.2.0-676.gc824a7fa.el9.x86_ 943 kB/s | 5.9 MB 00:06 2026-02-25T14:20:42.335 INFO:teuthology.orchestra.run.trial090.stdout:(2/37): ceph-selinux-20.2.0-676.gc824a7fa.el9.x 447 kB/s | 25 kB 00:00 2026-02-25T14:20:42.383 INFO:teuthology.orchestra.run.trial189.stdout:(14/37): mailcap-2.1.49-5.el9.noarch.rpm 81 kB/s | 33 kB 00:00 2026-02-25T14:20:42.391 INFO:teuthology.orchestra.run.trial090.stdout:(3/37): libcephfs-daemon-20.2.0-676.gc824a7fa.e 632 kB/s | 35 kB 00:00 2026-02-25T14:20:42.455 INFO:teuthology.orchestra.run.trial090.stdout:(4/37): libcephfs-proxy2-20.2.0-676.gc824a7fa.e 375 kB/s | 24 kB 00:00 2026-02-25T14:20:42.642 INFO:teuthology.orchestra.run.trial093.stdout:(35/37): librados2-20.2.0-676.gc824a7fa.el9.x86 5.9 MB/s | 3.5 MB 00:00 2026-02-25T14:20:42.926 INFO:teuthology.orchestra.run.trial093.stdout:(36/37): librbd1-20.2.0-676.gc824a7fa.el9.x86_6 3.7 MB/s | 2.8 MB 00:00 2026-02-25T14:20:43.264 INFO:teuthology.orchestra.run.trial090.stdout:(5/37): libcephfs2-20.2.0-676.gc824a7fa.el9.x86 1.1 MB/s | 869 kB 00:00 2026-02-25T14:20:43.382 INFO:teuthology.orchestra.run.trial090.stdout:(6/37): libradosstriper1-20.2.0-676.gc824a7fa.e 2.1 MB/s | 250 kB 00:00 2026-02-25T14:20:44.952 INFO:teuthology.orchestra.run.trial189.stdout:(15/37): nvme-cli-2.16-1.el9.x86_64.rpm 459 kB/s | 1.2 MB 00:02 2026-02-25T14:20:45.059 INFO:teuthology.orchestra.run.trial090.stdout:(7/37): ceph-radosgw-20.2.0-676.gc824a7fa.el9.x 2.6 MB/s | 24 MB 00:09 2026-02-25T14:20:45.085 INFO:teuthology.orchestra.run.trial090.stdout:(8/37): python3-ceph-argparse-20.2.0-676.gc824a 1.7 MB/s | 45 kB 00:00 2026-02-25T14:20:45.112 INFO:teuthology.orchestra.run.trial189.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 583 kB/s | 93 kB 00:00 2026-02-25T14:20:45.141 INFO:teuthology.orchestra.run.trial090.stdout:(9/37): python3-ceph-common-20.2.0-676.gc824a7f 3.1 MB/s | 175 kB 00:00 2026-02-25T14:20:45.158 INFO:teuthology.orchestra.run.trial189.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 2.2 MB/s | 104 kB 00:00 2026-02-25T14:20:45.171 INFO:teuthology.orchestra.run.trial090.stdout:(10/37): python3-cephfs-20.2.0-676.gc824a7fa.el 5.3 MB/s | 163 kB 00:00 2026-02-25T14:20:45.175 INFO:teuthology.orchestra.run.trial189.stdout:(18/37): libnbd-1.20.3-4.el9.x86_64.rpm 9.2 MB/s | 164 kB 00:00 2026-02-25T14:20:45.187 INFO:teuthology.orchestra.run.trial189.stdout:(19/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 14 MB/s | 160 kB 00:00 2026-02-25T14:20:45.196 INFO:teuthology.orchestra.run.trial189.stdout:(20/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 5.0 MB/s | 45 kB 00:00 2026-02-25T14:20:45.217 INFO:teuthology.orchestra.run.trial189.stdout:(21/37): librdkafka-1.6.1-102.el9.x86_64.rpm 31 MB/s | 662 kB 00:00 2026-02-25T14:20:45.228 INFO:teuthology.orchestra.run.trial189.stdout:(22/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 26 MB/s | 292 kB 00:00 2026-02-25T14:20:45.239 INFO:teuthology.orchestra.run.trial189.stdout:(23/37): qatlib-25.08.0-2.el9.x86_64.rpm 22 MB/s | 240 kB 00:00 2026-02-25T14:20:45.248 INFO:teuthology.orchestra.run.trial189.stdout:(24/37): qatlib-service-25.08.0-2.el9.x86_64.rp 4.1 MB/s | 37 kB 00:00 2026-02-25T14:20:45.257 INFO:teuthology.orchestra.run.trial189.stdout:(25/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 7.1 MB/s | 66 kB 00:00 2026-02-25T14:20:45.270 INFO:teuthology.orchestra.run.trial094.stdout:(37/37): ceph-radosgw-20.2.0-676.gc824a7fa.el9. 2.7 MB/s | 24 MB 00:08 2026-02-25T14:20:45.275 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:20:45.275 INFO:teuthology.orchestra.run.trial094.stdout:Total 8.7 MB/s | 79 MB 00:09 2026-02-25T14:20:45.293 INFO:teuthology.orchestra.run.trial090.stdout:(11/37): python3-rados-20.2.0-676.gc824a7fa.el9 2.6 MB/s | 324 kB 00:00 2026-02-25T14:20:45.350 INFO:teuthology.orchestra.run.trial090.stdout:(12/37): python3-rbd-20.2.0-676.gc824a7fa.el9.x 5.3 MB/s | 305 kB 00:00 2026-02-25T14:20:45.374 INFO:teuthology.orchestra.run.trial189.stdout:(26/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.6 MB/s | 308 kB 00:00 2026-02-25T14:20:45.376 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:20:45.396 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:20:45.397 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:20:45.402 INFO:teuthology.orchestra.run.trial090.stdout:(13/37): python3-rgw-20.2.0-676.gc824a7fa.el9.x 1.9 MB/s | 99 kB 00:00 2026-02-25T14:20:45.470 INFO:teuthology.orchestra.run.trial090.stdout:(14/37): mailcap-2.1.49-5.el9.noarch.rpm 491 kB/s | 33 kB 00:00 2026-02-25T14:20:45.496 INFO:teuthology.orchestra.run.trial189.stdout:(27/37): libarrow-9.0.0-15.el9.x86_64.rpm 36 MB/s | 4.4 MB 00:00 2026-02-25T14:20:45.524 INFO:teuthology.orchestra.run.trial189.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 892 kB/s | 25 kB 00:00 2026-02-25T14:20:45.554 INFO:teuthology.orchestra.run.trial189.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 1.6 MB/s | 49 kB 00:00 2026-02-25T14:20:45.576 INFO:teuthology.orchestra.run.trial090.stdout:(15/37): nvme-cli-2.16-1.el9.x86_64.rpm 11 MB/s | 1.2 MB 00:00 2026-02-25T14:20:45.583 INFO:teuthology.orchestra.run.trial189.stdout:(30/37): libunwind-1.6.2-1.el9.x86_64.rpm 2.3 MB/s | 67 kB 00:00 2026-02-25T14:20:45.599 INFO:teuthology.orchestra.run.trial090.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 4.0 MB/s | 93 kB 00:00 2026-02-25T14:20:45.625 INFO:teuthology.orchestra.run.trial189.stdout:(31/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 19 MB/s | 838 kB 00:00 2026-02-25T14:20:45.653 INFO:teuthology.orchestra.run.trial189.stdout:(32/37): re2-20211101-20.el9.x86_64.rpm 6.8 MB/s | 191 kB 00:00 2026-02-25T14:20:45.695 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:20:45.695 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:20:45.701 INFO:teuthology.orchestra.run.trial189.stdout:(33/37): thrift-0.15.0-4.el9.x86_64.rpm 33 MB/s | 1.6 MB 00:00 2026-02-25T14:20:45.728 INFO:teuthology.orchestra.run.trial090.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 809 kB/s | 104 kB 00:00 2026-02-25T14:20:45.784 INFO:teuthology.orchestra.run.trial090.stdout:(18/37): libnbd-1.20.3-4.el9.x86_64.rpm 2.9 MB/s | 164 kB 00:00 2026-02-25T14:20:45.819 INFO:teuthology.orchestra.run.trial090.stdout:(19/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.5 MB/s | 160 kB 00:00 2026-02-25T14:20:45.846 INFO:teuthology.orchestra.run.trial090.stdout:(20/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.6 MB/s | 45 kB 00:00 2026-02-25T14:20:45.914 INFO:teuthology.orchestra.run.trial090.stdout:(21/37): librdkafka-1.6.1-102.el9.x86_64.rpm 9.5 MB/s | 662 kB 00:00 2026-02-25T14:20:45.944 INFO:teuthology.orchestra.run.trial090.stdout:(22/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 9.5 MB/s | 292 kB 00:00 2026-02-25T14:20:45.973 INFO:teuthology.orchestra.run.trial090.stdout:(23/37): qatlib-25.08.0-2.el9.x86_64.rpm 8.0 MB/s | 240 kB 00:00 2026-02-25T14:20:45.999 INFO:teuthology.orchestra.run.trial090.stdout:(24/37): qatlib-service-25.08.0-2.el9.x86_64.rp 1.4 MB/s | 37 kB 00:00 2026-02-25T14:20:46.024 INFO:teuthology.orchestra.run.trial090.stdout:(25/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.5 MB/s | 66 kB 00:00 2026-02-25T14:20:46.044 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:20:46.056 INFO:teuthology.orchestra.run.trial094.stdout: Installing : thrift-0.15.0-4.el9.x86_64 1/39 2026-02-25T14:20:46.169 INFO:teuthology.orchestra.run.trial090.stdout:(26/37): librgw2-20.2.0-676.gc824a7fa.el9.x86_6 2.3 MB/s | 6.4 MB 00:02 2026-02-25T14:20:46.208 INFO:teuthology.orchestra.run.trial094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 2/39 2026-02-25T14:20:46.211 INFO:teuthology.orchestra.run.trial090.stdout:(27/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.6 MB/s | 308 kB 00:00 2026-02-25T14:20:46.212 INFO:teuthology.orchestra.run.trial094.stdout: Upgrading : librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 3/39 2026-02-25T14:20:46.262 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 3/39 2026-02-25T14:20:46.263 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 4/39 2026-02-25T14:20:46.274 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 4/39 2026-02-25T14:20:46.277 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libcephfs-daemon-2:20.2.0-676.gc824a7fa.el9.x86_64 5/39 2026-02-25T14:20:46.277 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 6/39 2026-02-25T14:20:46.283 INFO:teuthology.orchestra.run.trial090.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 346 kB/s | 25 kB 00:00 2026-02-25T14:20:46.303 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 6/39 2026-02-25T14:20:46.305 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 7/39 2026-02-25T14:20:46.310 INFO:teuthology.orchestra.run.trial090.stdout:(29/37): libarrow-9.0.0-15.el9.x86_64.rpm 31 MB/s | 4.4 MB 00:00 2026-02-25T14:20:46.311 INFO:teuthology.orchestra.run.trial090.stdout:(30/37): liboath-2.6.12-1.el9.x86_64.rpm 1.7 MB/s | 49 kB 00:00 2026-02-25T14:20:46.313 INFO:teuthology.orchestra.run.trial094.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/39 2026-02-25T14:20:46.315 INFO:teuthology.orchestra.run.trial094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/39 2026-02-25T14:20:46.318 INFO:teuthology.orchestra.run.trial094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 10/39 2026-02-25T14:20:46.322 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 11/39 2026-02-25T14:20:46.331 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-ceph-argparse-2:20.2.0-676.gc824a7fa.el9.x 12/39 2026-02-25T14:20:46.335 INFO:teuthology.orchestra.run.trial090.stdout:(31/37): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2026-02-25T14:20:46.339 INFO:teuthology.orchestra.run.trial090.stdout:(32/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 29 MB/s | 838 kB 00:00 2026-02-25T14:20:46.343 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 13/39 2026-02-25T14:20:46.344 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 14/39 2026-02-25T14:20:46.356 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 14/39 2026-02-25T14:20:46.361 INFO:teuthology.orchestra.run.trial090.stdout:(33/37): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2026-02-25T14:20:46.371 INFO:teuthology.orchestra.run.trial090.stdout:(34/37): thrift-0.15.0-4.el9.x86_64.rpm 50 MB/s | 1.6 MB 00:00 2026-02-25T14:20:46.387 INFO:teuthology.orchestra.run.trial094.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-25T14:20:46.392 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-25T14:20:46.398 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-25T14:20:46.400 INFO:teuthology.orchestra.run.trial094.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-25T14:20:46.404 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-25T14:20:46.530 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T14:20:46.531 INFO:teuthology.orchestra.run.trial094.stdout: Upgrading : librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 21/39 2026-02-25T14:20:46.556 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 21/39 2026-02-25T14:20:46.559 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 22/39 2026-02-25T14:20:46.565 INFO:teuthology.orchestra.run.trial094.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-25T14:20:46.747 INFO:teuthology.orchestra.run.trial090.stdout:(35/37): ceph-common-20.2.0-676.gc824a7fa.el9.x 2.2 MB/s | 24 MB 00:10 2026-02-25T14:20:46.817 INFO:teuthology.orchestra.run.trial189.stdout:(34/37): librados2-20.2.0-676.gc824a7fa.el9.x86 3.2 MB/s | 3.5 MB 00:01 2026-02-25T14:20:46.847 INFO:teuthology.orchestra.run.trial094.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-25T14:20:46.848 INFO:teuthology.orchestra.run.trial094.stdout: Installing : librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 25/39 2026-02-25T14:20:46.864 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 25/39 2026-02-25T14:20:46.868 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 26/39 2026-02-25T14:20:46.873 INFO:teuthology.orchestra.run.trial094.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-25T14:20:46.875 INFO:teuthology.orchestra.run.trial094.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-25T14:20:46.876 INFO:teuthology.orchestra.run.trial094.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T14:20:46.891 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T14:20:47.005 INFO:teuthology.orchestra.run.trial094.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-25T14:20:47.007 INFO:teuthology.orchestra.run.trial094.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T14:20:47.280 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T14:20:47.281 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-25T14:20:47.281 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:20:47.326 INFO:teuthology.orchestra.run.trial094.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-25T14:20:47.329 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-ceph-common-2:20.2.0-676.gc824a7fa.el9.x86 33/39 2026-02-25T14:20:47.428 INFO:teuthology.orchestra.run.trial189.stdout:(35/37): ceph-radosgw-20.2.0-676.gc824a7fa.el9. 2.2 MB/s | 24 MB 00:10 2026-02-25T14:20:47.497 INFO:teuthology.orchestra.run.trial090.stdout:(36/37): librbd1-20.2.0-676.gc824a7fa.el9.x86_6 2.5 MB/s | 2.8 MB 00:01 2026-02-25T14:20:47.523 INFO:teuthology.orchestra.run.trial189.stdout:(36/37): ceph-common-20.2.0-676.gc824a7fa.el9.x 2.2 MB/s | 24 MB 00:10 2026-02-25T14:20:47.553 INFO:teuthology.orchestra.run.trial189.stdout:(37/37): librbd1-20.2.0-676.gc824a7fa.el9.x86_6 3.9 MB/s | 2.8 MB 00:00 2026-02-25T14:20:47.556 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:20:47.556 INFO:teuthology.orchestra.run.trial189.stdout:Total 6.8 MB/s | 79 MB 00:11 2026-02-25T14:20:47.654 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:20:47.672 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:20:47.672 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:20:47.740 INFO:teuthology.orchestra.run.trial090.stdout:(37/37): librados2-20.2.0-676.gc824a7fa.el9.x86 2.5 MB/s | 3.5 MB 00:01 2026-02-25T14:20:47.743 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:20:47.743 INFO:teuthology.orchestra.run.trial090.stdout:Total 6.4 MB/s | 79 MB 00:12 2026-02-25T14:20:47.840 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:20:47.858 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:20:47.858 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:20:47.969 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:20:47.969 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:20:48.157 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:20:48.157 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:20:48.320 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:20:48.331 INFO:teuthology.orchestra.run.trial189.stdout: Installing : thrift-0.15.0-4.el9.x86_64 1/39 2026-02-25T14:20:48.404 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 34/39 2026-02-25T14:20:48.407 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 34/39 2026-02-25T14:20:48.485 INFO:teuthology.orchestra.run.trial189.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 2/39 2026-02-25T14:20:48.488 INFO:teuthology.orchestra.run.trial189.stdout: Upgrading : librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 3/39 2026-02-25T14:20:48.500 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:20:48.511 INFO:teuthology.orchestra.run.trial090.stdout: Installing : thrift-0.15.0-4.el9.x86_64 1/39 2026-02-25T14:20:48.538 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 3/39 2026-02-25T14:20:48.539 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 4/39 2026-02-25T14:20:48.552 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 4/39 2026-02-25T14:20:48.554 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libcephfs-daemon-2:20.2.0-676.gc824a7fa.el9.x86_64 5/39 2026-02-25T14:20:48.554 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 6/39 2026-02-25T14:20:48.580 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 6/39 2026-02-25T14:20:48.582 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 7/39 2026-02-25T14:20:48.590 INFO:teuthology.orchestra.run.trial189.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/39 2026-02-25T14:20:48.592 INFO:teuthology.orchestra.run.trial189.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/39 2026-02-25T14:20:48.595 INFO:teuthology.orchestra.run.trial189.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 10/39 2026-02-25T14:20:48.599 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 11/39 2026-02-25T14:20:48.608 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-ceph-argparse-2:20.2.0-676.gc824a7fa.el9.x 12/39 2026-02-25T14:20:48.620 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 13/39 2026-02-25T14:20:48.621 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 14/39 2026-02-25T14:20:48.633 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 14/39 2026-02-25T14:20:48.664 INFO:teuthology.orchestra.run.trial189.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-25T14:20:48.667 INFO:teuthology.orchestra.run.trial090.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 2/39 2026-02-25T14:20:48.669 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-25T14:20:48.671 INFO:teuthology.orchestra.run.trial090.stdout: Upgrading : librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 3/39 2026-02-25T14:20:48.675 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-25T14:20:48.678 INFO:teuthology.orchestra.run.trial189.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-25T14:20:48.682 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-25T14:20:48.706 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 34/39 2026-02-25T14:20:48.709 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 35/39 2026-02-25T14:20:48.719 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 3/39 2026-02-25T14:20:48.720 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 4/39 2026-02-25T14:20:48.733 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 4/39 2026-02-25T14:20:48.737 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libcephfs-daemon-2:20.2.0-676.gc824a7fa.el9.x86_64 5/39 2026-02-25T14:20:48.737 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 6/39 2026-02-25T14:20:48.750 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 35/39 2026-02-25T14:20:48.750 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-25T14:20:48.750 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-25T14:20:48.750 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:20:48.754 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 36/39 2026-02-25T14:20:48.763 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 6/39 2026-02-25T14:20:48.765 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 7/39 2026-02-25T14:20:48.772 INFO:teuthology.orchestra.run.trial090.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/39 2026-02-25T14:20:48.775 INFO:teuthology.orchestra.run.trial090.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/39 2026-02-25T14:20:48.778 INFO:teuthology.orchestra.run.trial090.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 10/39 2026-02-25T14:20:48.782 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 11/39 2026-02-25T14:20:48.791 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-ceph-argparse-2:20.2.0-676.gc824a7fa.el9.x 12/39 2026-02-25T14:20:48.803 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 13/39 2026-02-25T14:20:48.803 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T14:20:48.804 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 14/39 2026-02-25T14:20:48.805 INFO:teuthology.orchestra.run.trial189.stdout: Upgrading : librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 21/39 2026-02-25T14:20:48.808 INFO:teuthology.orchestra.run.trial093.stdout:(37/37): ceph-common-20.2.0-676.gc824a7fa.el9.x 1.8 MB/s | 24 MB 00:13 2026-02-25T14:20:48.812 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:20:48.812 INFO:teuthology.orchestra.run.trial093.stdout:Total 5.9 MB/s | 79 MB 00:13 2026-02-25T14:20:48.817 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 14/39 2026-02-25T14:20:48.830 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 21/39 2026-02-25T14:20:48.832 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 22/39 2026-02-25T14:20:48.838 INFO:teuthology.orchestra.run.trial189.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-25T14:20:48.850 INFO:teuthology.orchestra.run.trial090.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-25T14:20:48.855 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-25T14:20:48.861 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-25T14:20:48.863 INFO:teuthology.orchestra.run.trial090.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-25T14:20:48.867 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-25T14:20:48.911 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:20:48.930 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:20:48.930 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:20:48.992 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T14:20:48.993 INFO:teuthology.orchestra.run.trial090.stdout: Upgrading : librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 21/39 2026-02-25T14:20:49.019 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 21/39 2026-02-25T14:20:49.021 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 22/39 2026-02-25T14:20:49.027 INFO:teuthology.orchestra.run.trial090.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-25T14:20:49.114 INFO:teuthology.orchestra.run.trial189.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-25T14:20:49.115 INFO:teuthology.orchestra.run.trial189.stdout: Installing : librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 25/39 2026-02-25T14:20:49.132 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 25/39 2026-02-25T14:20:49.135 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 26/39 2026-02-25T14:20:49.140 INFO:teuthology.orchestra.run.trial189.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-25T14:20:49.142 INFO:teuthology.orchestra.run.trial189.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-25T14:20:49.143 INFO:teuthology.orchestra.run.trial189.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T14:20:49.160 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T14:20:49.228 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:20:49.228 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:20:49.275 INFO:teuthology.orchestra.run.trial189.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-25T14:20:49.277 INFO:teuthology.orchestra.run.trial189.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T14:20:49.312 INFO:teuthology.orchestra.run.trial090.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-25T14:20:49.313 INFO:teuthology.orchestra.run.trial090.stdout: Installing : librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 25/39 2026-02-25T14:20:49.329 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 25/39 2026-02-25T14:20:49.332 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 26/39 2026-02-25T14:20:49.337 INFO:teuthology.orchestra.run.trial090.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-25T14:20:49.339 INFO:teuthology.orchestra.run.trial090.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-25T14:20:49.340 INFO:teuthology.orchestra.run.trial090.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T14:20:49.357 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T14:20:49.470 INFO:teuthology.orchestra.run.trial090.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-25T14:20:49.473 INFO:teuthology.orchestra.run.trial090.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T14:20:49.550 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T14:20:49.550 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-25T14:20:49.550 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:20:49.584 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:20:49.593 INFO:teuthology.orchestra.run.trial189.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-25T14:20:49.595 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-ceph-common-2:20.2.0-676.gc824a7fa.el9.x86 33/39 2026-02-25T14:20:49.596 INFO:teuthology.orchestra.run.trial093.stdout: Installing : thrift-0.15.0-4.el9.x86_64 1/39 2026-02-25T14:20:49.750 INFO:teuthology.orchestra.run.trial093.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 2/39 2026-02-25T14:20:49.753 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T14:20:49.753 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-25T14:20:49.753 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:20:49.754 INFO:teuthology.orchestra.run.trial093.stdout: Upgrading : librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 3/39 2026-02-25T14:20:49.795 INFO:teuthology.orchestra.run.trial090.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-25T14:20:49.797 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-ceph-common-2:20.2.0-676.gc824a7fa.el9.x86 33/39 2026-02-25T14:20:49.802 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 3/39 2026-02-25T14:20:49.803 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 4/39 2026-02-25T14:20:49.815 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 4/39 2026-02-25T14:20:49.817 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libcephfs-daemon-2:20.2.0-676.gc824a7fa.el9.x86_64 5/39 2026-02-25T14:20:49.818 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 6/39 2026-02-25T14:20:49.843 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 6/39 2026-02-25T14:20:49.847 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 7/39 2026-02-25T14:20:49.854 INFO:teuthology.orchestra.run.trial093.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/39 2026-02-25T14:20:49.857 INFO:teuthology.orchestra.run.trial093.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/39 2026-02-25T14:20:49.859 INFO:teuthology.orchestra.run.trial093.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 10/39 2026-02-25T14:20:49.863 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 11/39 2026-02-25T14:20:49.872 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-ceph-argparse-2:20.2.0-676.gc824a7fa.el9.x 12/39 2026-02-25T14:20:49.884 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 13/39 2026-02-25T14:20:49.884 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 14/39 2026-02-25T14:20:49.898 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 14/39 2026-02-25T14:20:49.930 INFO:teuthology.orchestra.run.trial093.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-25T14:20:49.935 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-25T14:20:49.942 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-25T14:20:49.944 INFO:teuthology.orchestra.run.trial093.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-25T14:20:49.948 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-25T14:20:50.071 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T14:20:50.072 INFO:teuthology.orchestra.run.trial093.stdout: Upgrading : librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 21/39 2026-02-25T14:20:50.100 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 21/39 2026-02-25T14:20:50.103 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 22/39 2026-02-25T14:20:50.109 INFO:teuthology.orchestra.run.trial093.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-25T14:20:50.387 INFO:teuthology.orchestra.run.trial093.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-25T14:20:50.388 INFO:teuthology.orchestra.run.trial093.stdout: Installing : librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 25/39 2026-02-25T14:20:50.405 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 25/39 2026-02-25T14:20:50.408 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 26/39 2026-02-25T14:20:50.413 INFO:teuthology.orchestra.run.trial093.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-25T14:20:50.415 INFO:teuthology.orchestra.run.trial093.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-25T14:20:50.416 INFO:teuthology.orchestra.run.trial093.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T14:20:50.434 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T14:20:50.549 INFO:teuthology.orchestra.run.trial093.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-25T14:20:50.551 INFO:teuthology.orchestra.run.trial093.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T14:20:50.669 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 34/39 2026-02-25T14:20:50.671 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 34/39 2026-02-25T14:20:50.828 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T14:20:50.828 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-25T14:20:50.828 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:20:50.873 INFO:teuthology.orchestra.run.trial093.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-25T14:20:50.875 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-ceph-common-2:20.2.0-676.gc824a7fa.el9.x86 33/39 2026-02-25T14:20:50.894 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 34/39 2026-02-25T14:20:50.896 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 34/39 2026-02-25T14:20:50.964 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 34/39 2026-02-25T14:20:50.967 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 35/39 2026-02-25T14:20:51.004 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 35/39 2026-02-25T14:20:51.004 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-25T14:20:51.004 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-25T14:20:51.005 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:20:51.008 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 36/39 2026-02-25T14:20:51.198 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 34/39 2026-02-25T14:20:51.201 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 35/39 2026-02-25T14:20:51.244 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 35/39 2026-02-25T14:20:51.244 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-25T14:20:51.244 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-25T14:20:51.244 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:20:51.247 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 36/39 2026-02-25T14:20:51.952 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 34/39 2026-02-25T14:20:51.955 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 34/39 2026-02-25T14:20:52.249 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 34/39 2026-02-25T14:20:52.252 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 35/39 2026-02-25T14:20:52.292 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 35/39 2026-02-25T14:20:52.292 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-25T14:20:52.293 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-25T14:20:52.293 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:20:52.295 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 36/39 2026-02-25T14:20:53.245 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 36/39 2026-02-25T14:20:53.245 INFO:teuthology.orchestra.run.trial094.stdout:skipping the directory /sys 2026-02-25T14:20:53.245 INFO:teuthology.orchestra.run.trial094.stdout:skipping the directory /proc 2026-02-25T14:20:53.245 INFO:teuthology.orchestra.run.trial094.stdout:skipping the directory /mnt 2026-02-25T14:20:53.245 INFO:teuthology.orchestra.run.trial094.stdout:skipping the directory /var/tmp 2026-02-25T14:20:53.245 INFO:teuthology.orchestra.run.trial094.stdout:skipping the directory /home 2026-02-25T14:20:53.245 INFO:teuthology.orchestra.run.trial094.stdout:skipping the directory /root 2026-02-25T14:20:53.245 INFO:teuthology.orchestra.run.trial094.stdout:skipping the directory /tmp 2026-02-25T14:20:53.246 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:20:54.272 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 37/39 2026-02-25T14:20:54.295 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 37/39 2026-02-25T14:20:54.295 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:20:54.295 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-25T14:20:54.295 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T14:20:54.295 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T14:20:54.295 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:20:54.296 INFO:teuthology.orchestra.run.trial094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T14:20:54.308 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T14:20:54.309 INFO:teuthology.orchestra.run.trial094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T14:20:54.364 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T14:20:54.364 INFO:teuthology.orchestra.run.trial094.stdout:Creating group 'qat' with GID 992. 2026-02-25T14:20:54.364 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:20:54.689 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 1/39 2026-02-25T14:20:54.689 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 2/39 2026-02-25T14:20:54.689 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 3/39 2026-02-25T14:20:54.690 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 4/39 2026-02-25T14:20:54.690 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libcephfs-daemon-2:20.2.0-676.gc824a7fa.el9.x86_64 5/39 2026-02-25T14:20:54.690 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 6/39 2026-02-25T14:20:54.690 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 7/39 2026-02-25T14:20:54.690 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 8/39 2026-02-25T14:20:54.690 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 9/39 2026-02-25T14:20:54.691 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-ceph-argparse-2:20.2.0-676.gc824a7fa.el9.x 10/39 2026-02-25T14:20:54.691 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-ceph-common-2:20.2.0-676.gc824a7fa.el9.x86 11/39 2026-02-25T14:20:54.691 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 12/39 2026-02-25T14:20:54.691 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 13/39 2026-02-25T14:20:54.691 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 14/39 2026-02-25T14:20:54.691 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 15/39 2026-02-25T14:20:54.691 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-25T14:20:54.691 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-25T14:20:54.691 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-25T14:20:54.691 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-25T14:20:54.691 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T14:20:54.691 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-25T14:20:54.692 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-25T14:20:54.692 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-25T14:20:54.692 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-25T14:20:54.692 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-25T14:20:54.692 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-25T14:20:54.692 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-25T14:20:54.692 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-25T14:20:54.692 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-25T14:20:54.692 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-25T14:20:54.692 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-25T14:20:54.692 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-25T14:20:54.692 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-25T14:20:54.692 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-25T14:20:54.693 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-25T14:20:54.693 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 36/39 2026-02-25T14:20:54.693 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-25T14:20:54.693 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 38/39 2026-02-25T14:20:54.755 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T14:20:54.756 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:20:54.756 INFO:teuthology.orchestra.run.trial094.stdout:Upgraded: 2026-02-25T14:20:54.756 INFO:teuthology.orchestra.run.trial094.stdout: librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.756 INFO:teuthology.orchestra.run.trial094.stdout: librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.756 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:20:54.756 INFO:teuthology.orchestra.run.trial094.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-25T14:20:54.756 INFO:teuthology.orchestra.run.trial094.stdout: ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.757 INFO:teuthology.orchestra.run.trial094.stdout: ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.757 INFO:teuthology.orchestra.run.trial094.stdout: ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.757 INFO:teuthology.orchestra.run.trial094.stdout: ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.757 INFO:teuthology.orchestra.run.trial094.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-25T14:20:54.757 INFO:teuthology.orchestra.run.trial094.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-25T14:20:54.757 INFO:teuthology.orchestra.run.trial094.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-25T14:20:54.757 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs-daemon-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.757 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.757 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.757 INFO:teuthology.orchestra.run.trial094.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-25T14:20:54.758 INFO:teuthology.orchestra.run.trial094.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-25T14:20:54.758 INFO:teuthology.orchestra.run.trial094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-25T14:20:54.758 INFO:teuthology.orchestra.run.trial094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-25T14:20:54.758 INFO:teuthology.orchestra.run.trial094.stdout: libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.758 INFO:teuthology.orchestra.run.trial094.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-25T14:20:54.758 INFO:teuthology.orchestra.run.trial094.stdout: librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.758 INFO:teuthology.orchestra.run.trial094.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-25T14:20:54.758 INFO:teuthology.orchestra.run.trial094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-25T14:20:54.758 INFO:teuthology.orchestra.run.trial094.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-25T14:20:54.759 INFO:teuthology.orchestra.run.trial094.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-25T14:20:54.759 INFO:teuthology.orchestra.run.trial094.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-25T14:20:54.759 INFO:teuthology.orchestra.run.trial094.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-25T14:20:54.759 INFO:teuthology.orchestra.run.trial094.stdout: python3-ceph-argparse-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.759 INFO:teuthology.orchestra.run.trial094.stdout: python3-ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.759 INFO:teuthology.orchestra.run.trial094.stdout: python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.759 INFO:teuthology.orchestra.run.trial094.stdout: python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.759 INFO:teuthology.orchestra.run.trial094.stdout: python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.759 INFO:teuthology.orchestra.run.trial094.stdout: python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:54.759 INFO:teuthology.orchestra.run.trial094.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-25T14:20:54.759 INFO:teuthology.orchestra.run.trial094.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-25T14:20:54.759 INFO:teuthology.orchestra.run.trial094.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-25T14:20:54.760 INFO:teuthology.orchestra.run.trial094.stdout: re2-1:20211101-20.el9.x86_64 2026-02-25T14:20:54.760 INFO:teuthology.orchestra.run.trial094.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-25T14:20:54.760 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:20:54.760 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:20:54.800 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-test 2026-02-25T14:20:55.114 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:21 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:20:55.197 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:20:55.198 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:20:55.198 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repository Size 2026-02-25T14:20:55.198 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:20:55.198 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:20:55.198 INFO:teuthology.orchestra.run.trial094.stdout: ceph-test x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 84 M 2026-02-25T14:20:55.198 INFO:teuthology.orchestra.run.trial094.stdout:Installing dependencies: 2026-02-25T14:20:55.198 INFO:teuthology.orchestra.run.trial094.stdout: libcephsqlite x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 164 k 2026-02-25T14:20:55.198 INFO:teuthology.orchestra.run.trial094.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-25T14:20:55.198 INFO:teuthology.orchestra.run.trial094.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-25T14:20:55.199 INFO:teuthology.orchestra.run.trial094.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-25T14:20:55.199 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:20:55.199 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:20:55.199 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:20:55.199 INFO:teuthology.orchestra.run.trial094.stdout:Install 5 Packages 2026-02-25T14:20:55.199 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:20:55.199 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 85 M 2026-02-25T14:20:55.199 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 365 M 2026-02-25T14:20:55.199 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:20:55.511 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 36/39 2026-02-25T14:20:55.512 INFO:teuthology.orchestra.run.trial189.stdout:skipping the directory /sys 2026-02-25T14:20:55.512 INFO:teuthology.orchestra.run.trial189.stdout:skipping the directory /proc 2026-02-25T14:20:55.512 INFO:teuthology.orchestra.run.trial189.stdout:skipping the directory /mnt 2026-02-25T14:20:55.512 INFO:teuthology.orchestra.run.trial189.stdout:skipping the directory /var/tmp 2026-02-25T14:20:55.512 INFO:teuthology.orchestra.run.trial189.stdout:skipping the directory /home 2026-02-25T14:20:55.512 INFO:teuthology.orchestra.run.trial189.stdout:skipping the directory /root 2026-02-25T14:20:55.512 INFO:teuthology.orchestra.run.trial189.stdout:skipping the directory /tmp 2026-02-25T14:20:55.512 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:20:55.538 INFO:teuthology.orchestra.run.trial094.stdout:(1/5): libcephsqlite-20.2.0-676.gc824a7fa.el9.x 1.8 MB/s | 164 kB 00:00 2026-02-25T14:20:55.782 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 36/39 2026-02-25T14:20:55.782 INFO:teuthology.orchestra.run.trial090.stdout:skipping the directory /sys 2026-02-25T14:20:55.782 INFO:teuthology.orchestra.run.trial090.stdout:skipping the directory /proc 2026-02-25T14:20:55.782 INFO:teuthology.orchestra.run.trial090.stdout:skipping the directory /mnt 2026-02-25T14:20:55.782 INFO:teuthology.orchestra.run.trial090.stdout:skipping the directory /var/tmp 2026-02-25T14:20:55.783 INFO:teuthology.orchestra.run.trial090.stdout:skipping the directory /home 2026-02-25T14:20:55.783 INFO:teuthology.orchestra.run.trial090.stdout:skipping the directory /root 2026-02-25T14:20:55.783 INFO:teuthology.orchestra.run.trial090.stdout:skipping the directory /tmp 2026-02-25T14:20:55.783 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:20:56.201 INFO:teuthology.orchestra.run.trial094.stdout:(2/5): libxslt-1.1.34-12.el9.x86_64.rpm 309 kB/s | 233 kB 00:00 2026-02-25T14:20:56.301 INFO:teuthology.orchestra.run.trial094.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 640 kB/s | 64 kB 00:00 2026-02-25T14:20:56.340 INFO:teuthology.orchestra.run.trial094.stdout:(4/5): socat-1.7.4.1-8.el9.x86_64.rpm 378 kB/s | 303 kB 00:00 2026-02-25T14:20:56.539 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 37/39 2026-02-25T14:20:56.563 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 37/39 2026-02-25T14:20:56.563 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:20:56.563 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-25T14:20:56.563 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T14:20:56.563 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T14:20:56.564 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:20:56.565 INFO:teuthology.orchestra.run.trial189.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T14:20:56.578 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T14:20:56.578 INFO:teuthology.orchestra.run.trial189.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T14:20:56.636 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T14:20:56.637 INFO:teuthology.orchestra.run.trial189.stdout:Creating group 'qat' with GID 992. 2026-02-25T14:20:56.637 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:20:56.833 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 37/39 2026-02-25T14:20:56.848 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 36/39 2026-02-25T14:20:56.848 INFO:teuthology.orchestra.run.trial093.stdout:skipping the directory /sys 2026-02-25T14:20:56.848 INFO:teuthology.orchestra.run.trial093.stdout:skipping the directory /proc 2026-02-25T14:20:56.848 INFO:teuthology.orchestra.run.trial093.stdout:skipping the directory /mnt 2026-02-25T14:20:56.848 INFO:teuthology.orchestra.run.trial093.stdout:skipping the directory /var/tmp 2026-02-25T14:20:56.848 INFO:teuthology.orchestra.run.trial093.stdout:skipping the directory /home 2026-02-25T14:20:56.849 INFO:teuthology.orchestra.run.trial093.stdout:skipping the directory /root 2026-02-25T14:20:56.849 INFO:teuthology.orchestra.run.trial093.stdout:skipping the directory /tmp 2026-02-25T14:20:56.849 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:20:56.857 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 37/39 2026-02-25T14:20:56.857 INFO:teuthology.orchestra.run.trial090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:20:56.857 INFO:teuthology.orchestra.run.trial090.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-25T14:20:56.857 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T14:20:56.857 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T14:20:56.857 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:20:56.858 INFO:teuthology.orchestra.run.trial090.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T14:20:56.870 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T14:20:56.870 INFO:teuthology.orchestra.run.trial090.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T14:20:56.926 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T14:20:56.926 INFO:teuthology.orchestra.run.trial090.stdout:Creating group 'qat' with GID 992. 2026-02-25T14:20:56.926 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:20:56.968 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 1/39 2026-02-25T14:20:56.968 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 2/39 2026-02-25T14:20:56.968 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 3/39 2026-02-25T14:20:56.968 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 4/39 2026-02-25T14:20:56.969 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libcephfs-daemon-2:20.2.0-676.gc824a7fa.el9.x86_64 5/39 2026-02-25T14:20:56.969 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 6/39 2026-02-25T14:20:56.969 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 7/39 2026-02-25T14:20:56.969 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 8/39 2026-02-25T14:20:56.969 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 9/39 2026-02-25T14:20:56.969 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-ceph-argparse-2:20.2.0-676.gc824a7fa.el9.x 10/39 2026-02-25T14:20:56.969 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-ceph-common-2:20.2.0-676.gc824a7fa.el9.x86 11/39 2026-02-25T14:20:56.969 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 12/39 2026-02-25T14:20:56.969 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 13/39 2026-02-25T14:20:56.969 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 14/39 2026-02-25T14:20:56.969 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 15/39 2026-02-25T14:20:56.969 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-25T14:20:56.970 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-25T14:20:56.970 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-25T14:20:56.970 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-25T14:20:56.970 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T14:20:56.970 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-25T14:20:56.970 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-25T14:20:56.970 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-25T14:20:56.970 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-25T14:20:56.970 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-25T14:20:56.970 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-25T14:20:56.970 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-25T14:20:56.970 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-25T14:20:56.971 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-25T14:20:56.971 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-25T14:20:56.971 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-25T14:20:56.971 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-25T14:20:56.971 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-25T14:20:56.971 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-25T14:20:56.971 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-25T14:20:56.971 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 36/39 2026-02-25T14:20:56.971 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-25T14:20:56.971 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 38/39 2026-02-25T14:20:57.036 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T14:20:57.036 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:20:57.037 INFO:teuthology.orchestra.run.trial189.stdout:Upgraded: 2026-02-25T14:20:57.037 INFO:teuthology.orchestra.run.trial189.stdout: librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.037 INFO:teuthology.orchestra.run.trial189.stdout: librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.037 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:20:57.039 INFO:teuthology.orchestra.run.trial189.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-25T14:20:57.040 INFO:teuthology.orchestra.run.trial189.stdout: ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.040 INFO:teuthology.orchestra.run.trial189.stdout: ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.040 INFO:teuthology.orchestra.run.trial189.stdout: ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.040 INFO:teuthology.orchestra.run.trial189.stdout: ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.040 INFO:teuthology.orchestra.run.trial189.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-25T14:20:57.040 INFO:teuthology.orchestra.run.trial189.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-25T14:20:57.040 INFO:teuthology.orchestra.run.trial189.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-25T14:20:57.040 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs-daemon-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.040 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.040 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.041 INFO:teuthology.orchestra.run.trial189.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-25T14:20:57.041 INFO:teuthology.orchestra.run.trial189.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-25T14:20:57.041 INFO:teuthology.orchestra.run.trial189.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-25T14:20:57.041 INFO:teuthology.orchestra.run.trial189.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-25T14:20:57.041 INFO:teuthology.orchestra.run.trial189.stdout: libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.041 INFO:teuthology.orchestra.run.trial189.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-25T14:20:57.041 INFO:teuthology.orchestra.run.trial189.stdout: librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.041 INFO:teuthology.orchestra.run.trial189.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-25T14:20:57.041 INFO:teuthology.orchestra.run.trial189.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-25T14:20:57.041 INFO:teuthology.orchestra.run.trial189.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-25T14:20:57.041 INFO:teuthology.orchestra.run.trial189.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-25T14:20:57.041 INFO:teuthology.orchestra.run.trial189.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-25T14:20:57.042 INFO:teuthology.orchestra.run.trial189.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-25T14:20:57.042 INFO:teuthology.orchestra.run.trial189.stdout: python3-ceph-argparse-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.042 INFO:teuthology.orchestra.run.trial189.stdout: python3-ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.042 INFO:teuthology.orchestra.run.trial189.stdout: python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.042 INFO:teuthology.orchestra.run.trial189.stdout: python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.042 INFO:teuthology.orchestra.run.trial189.stdout: python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.042 INFO:teuthology.orchestra.run.trial189.stdout: python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.042 INFO:teuthology.orchestra.run.trial189.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-25T14:20:57.042 INFO:teuthology.orchestra.run.trial189.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-25T14:20:57.042 INFO:teuthology.orchestra.run.trial189.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-25T14:20:57.042 INFO:teuthology.orchestra.run.trial189.stdout: re2-1:20211101-20.el9.x86_64 2026-02-25T14:20:57.042 INFO:teuthology.orchestra.run.trial189.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-25T14:20:57.043 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:20:57.043 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:20:57.079 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-test 2026-02-25T14:20:57.239 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 1/39 2026-02-25T14:20:57.239 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 2/39 2026-02-25T14:20:57.239 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 3/39 2026-02-25T14:20:57.240 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 4/39 2026-02-25T14:20:57.240 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libcephfs-daemon-2:20.2.0-676.gc824a7fa.el9.x86_64 5/39 2026-02-25T14:20:57.240 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 6/39 2026-02-25T14:20:57.240 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 7/39 2026-02-25T14:20:57.240 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 8/39 2026-02-25T14:20:57.240 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 9/39 2026-02-25T14:20:57.240 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-ceph-argparse-2:20.2.0-676.gc824a7fa.el9.x 10/39 2026-02-25T14:20:57.240 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-ceph-common-2:20.2.0-676.gc824a7fa.el9.x86 11/39 2026-02-25T14:20:57.240 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 12/39 2026-02-25T14:20:57.240 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 13/39 2026-02-25T14:20:57.241 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 14/39 2026-02-25T14:20:57.241 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 15/39 2026-02-25T14:20:57.241 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-25T14:20:57.241 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-25T14:20:57.241 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-25T14:20:57.241 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-25T14:20:57.241 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T14:20:57.241 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-25T14:20:57.241 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-25T14:20:57.241 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-25T14:20:57.242 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-25T14:20:57.242 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-25T14:20:57.242 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-25T14:20:57.242 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-25T14:20:57.242 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-25T14:20:57.242 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-25T14:20:57.242 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-25T14:20:57.242 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-25T14:20:57.242 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-25T14:20:57.242 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-25T14:20:57.242 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-25T14:20:57.243 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-25T14:20:57.243 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 36/39 2026-02-25T14:20:57.243 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-25T14:20:57.243 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 38/39 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout:Upgraded: 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: libcephfs-daemon-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-25T14:20:57.307 INFO:teuthology.orchestra.run.trial090.stdout: libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: python3-ceph-argparse-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: python3-ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: re2-1:20211101-20.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:20:57.308 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:20:57.351 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-test 2026-02-25T14:20:57.402 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:23 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:20:57.488 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:20:57.489 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:20:57.489 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repository Size 2026-02-25T14:20:57.489 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:20:57.489 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:20:57.489 INFO:teuthology.orchestra.run.trial189.stdout: ceph-test x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 84 M 2026-02-25T14:20:57.489 INFO:teuthology.orchestra.run.trial189.stdout:Installing dependencies: 2026-02-25T14:20:57.489 INFO:teuthology.orchestra.run.trial189.stdout: libcephsqlite x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 164 k 2026-02-25T14:20:57.490 INFO:teuthology.orchestra.run.trial189.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-25T14:20:57.490 INFO:teuthology.orchestra.run.trial189.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-25T14:20:57.490 INFO:teuthology.orchestra.run.trial189.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-25T14:20:57.490 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:20:57.490 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:20:57.490 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:20:57.490 INFO:teuthology.orchestra.run.trial189.stdout:Install 5 Packages 2026-02-25T14:20:57.490 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:20:57.490 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 85 M 2026-02-25T14:20:57.490 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 365 M 2026-02-25T14:20:57.491 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:20:57.674 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:00:23 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:20:57.680 INFO:teuthology.orchestra.run.trial189.stdout:(1/5): libxslt-1.1.34-12.el9.x86_64.rpm 1.7 MB/s | 233 kB 00:00 2026-02-25T14:20:57.697 INFO:teuthology.orchestra.run.trial094.stdout:(5/5): ceph-test-20.2.0-676.gc824a7fa.el9.x86_6 37 MB/s | 84 MB 00:02 2026-02-25T14:20:57.700 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:20:57.700 INFO:teuthology.orchestra.run.trial094.stdout:Total 34 MB/s | 85 MB 00:02 2026-02-25T14:20:57.724 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:20:57.730 INFO:teuthology.orchestra.run.trial189.stdout:(2/5): socat-1.7.4.1-8.el9.x86_64.rpm 5.9 MB/s | 303 kB 00:00 2026-02-25T14:20:57.735 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:20:57.735 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:20:57.755 INFO:teuthology.orchestra.run.trial189.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.5 MB/s | 64 kB 00:00 2026-02-25T14:20:57.757 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:20:57.758 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:20:57.758 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repository Size 2026-02-25T14:20:57.758 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:20:57.759 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:20:57.759 INFO:teuthology.orchestra.run.trial090.stdout: ceph-test x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 84 M 2026-02-25T14:20:57.759 INFO:teuthology.orchestra.run.trial090.stdout:Installing dependencies: 2026-02-25T14:20:57.759 INFO:teuthology.orchestra.run.trial090.stdout: libcephsqlite x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 164 k 2026-02-25T14:20:57.759 INFO:teuthology.orchestra.run.trial090.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-25T14:20:57.759 INFO:teuthology.orchestra.run.trial090.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-25T14:20:57.759 INFO:teuthology.orchestra.run.trial090.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-25T14:20:57.759 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:20:57.759 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:20:57.759 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:20:57.759 INFO:teuthology.orchestra.run.trial090.stdout:Install 5 Packages 2026-02-25T14:20:57.760 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:20:57.760 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 85 M 2026-02-25T14:20:57.760 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 365 M 2026-02-25T14:20:57.760 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:20:57.878 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 37/39 2026-02-25T14:20:57.902 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 37/39 2026-02-25T14:20:57.902 INFO:teuthology.orchestra.run.trial093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:20:57.902 INFO:teuthology.orchestra.run.trial093.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-25T14:20:57.902 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T14:20:57.902 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T14:20:57.902 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:20:57.904 INFO:teuthology.orchestra.run.trial093.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T14:20:57.916 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T14:20:57.916 INFO:teuthology.orchestra.run.trial093.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T14:20:57.971 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T14:20:57.972 INFO:teuthology.orchestra.run.trial093.stdout:Creating group 'qat' with GID 992. 2026-02-25T14:20:57.972 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:20:57.979 INFO:teuthology.orchestra.run.trial189.stdout:(4/5): libcephsqlite-20.2.0-676.gc824a7fa.el9.x 382 kB/s | 164 kB 00:00 2026-02-25T14:20:58.026 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:20:58.026 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:20:58.293 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 1/39 2026-02-25T14:20:58.293 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 2/39 2026-02-25T14:20:58.294 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 3/39 2026-02-25T14:20:58.294 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 4/39 2026-02-25T14:20:58.294 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libcephfs-daemon-2:20.2.0-676.gc824a7fa.el9.x86_64 5/39 2026-02-25T14:20:58.294 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 6/39 2026-02-25T14:20:58.294 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 7/39 2026-02-25T14:20:58.294 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 8/39 2026-02-25T14:20:58.294 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 9/39 2026-02-25T14:20:58.294 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-ceph-argparse-2:20.2.0-676.gc824a7fa.el9.x 10/39 2026-02-25T14:20:58.294 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-ceph-common-2:20.2.0-676.gc824a7fa.el9.x86 11/39 2026-02-25T14:20:58.294 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 12/39 2026-02-25T14:20:58.294 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 13/39 2026-02-25T14:20:58.295 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 14/39 2026-02-25T14:20:58.295 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 15/39 2026-02-25T14:20:58.295 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-25T14:20:58.295 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-25T14:20:58.295 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-25T14:20:58.295 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-25T14:20:58.295 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T14:20:58.295 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-25T14:20:58.295 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-25T14:20:58.295 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-25T14:20:58.295 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-25T14:20:58.295 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-25T14:20:58.295 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-25T14:20:58.296 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-25T14:20:58.296 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-25T14:20:58.296 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-25T14:20:58.296 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-25T14:20:58.296 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-25T14:20:58.296 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-25T14:20:58.296 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-25T14:20:58.296 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-25T14:20:58.296 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-25T14:20:58.296 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 36/39 2026-02-25T14:20:58.296 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-25T14:20:58.296 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 38/39 2026-02-25T14:20:58.342 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:20:58.348 INFO:teuthology.orchestra.run.trial094.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-25T14:20:58.352 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-25T14:20:58.360 INFO:teuthology.orchestra.run.trial094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-25T14:20:58.361 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 4/5 2026-02-25T14:20:58.363 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T14:20:58.363 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:20:58.363 INFO:teuthology.orchestra.run.trial093.stdout:Upgraded: 2026-02-25T14:20:58.364 INFO:teuthology.orchestra.run.trial093.stdout: librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.364 INFO:teuthology.orchestra.run.trial093.stdout: librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.364 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:20:58.364 INFO:teuthology.orchestra.run.trial093.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-25T14:20:58.364 INFO:teuthology.orchestra.run.trial093.stdout: ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.364 INFO:teuthology.orchestra.run.trial093.stdout: ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.364 INFO:teuthology.orchestra.run.trial093.stdout: ceph-radosgw-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.364 INFO:teuthology.orchestra.run.trial093.stdout: ceph-selinux-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.365 INFO:teuthology.orchestra.run.trial093.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-25T14:20:58.365 INFO:teuthology.orchestra.run.trial093.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-25T14:20:58.365 INFO:teuthology.orchestra.run.trial093.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-25T14:20:58.365 INFO:teuthology.orchestra.run.trial093.stdout: libcephfs-daemon-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.365 INFO:teuthology.orchestra.run.trial093.stdout: libcephfs-proxy2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.365 INFO:teuthology.orchestra.run.trial093.stdout: libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.365 INFO:teuthology.orchestra.run.trial093.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-25T14:20:58.365 INFO:teuthology.orchestra.run.trial093.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-25T14:20:58.366 INFO:teuthology.orchestra.run.trial093.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-25T14:20:58.366 INFO:teuthology.orchestra.run.trial093.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-25T14:20:58.366 INFO:teuthology.orchestra.run.trial093.stdout: libradosstriper1-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.366 INFO:teuthology.orchestra.run.trial093.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-25T14:20:58.366 INFO:teuthology.orchestra.run.trial093.stdout: librgw2-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.366 INFO:teuthology.orchestra.run.trial093.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-25T14:20:58.366 INFO:teuthology.orchestra.run.trial093.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-25T14:20:58.366 INFO:teuthology.orchestra.run.trial093.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-25T14:20:58.367 INFO:teuthology.orchestra.run.trial093.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-25T14:20:58.367 INFO:teuthology.orchestra.run.trial093.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-25T14:20:58.367 INFO:teuthology.orchestra.run.trial093.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-25T14:20:58.367 INFO:teuthology.orchestra.run.trial093.stdout: python3-ceph-argparse-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.367 INFO:teuthology.orchestra.run.trial093.stdout: python3-ceph-common-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.367 INFO:teuthology.orchestra.run.trial093.stdout: python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.367 INFO:teuthology.orchestra.run.trial093.stdout: python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.367 INFO:teuthology.orchestra.run.trial093.stdout: python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.367 INFO:teuthology.orchestra.run.trial093.stdout: python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:20:58.367 INFO:teuthology.orchestra.run.trial093.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-25T14:20:58.368 INFO:teuthology.orchestra.run.trial093.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-25T14:20:58.368 INFO:teuthology.orchestra.run.trial093.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-25T14:20:58.368 INFO:teuthology.orchestra.run.trial093.stdout: re2-1:20211101-20.el9.x86_64 2026-02-25T14:20:58.368 INFO:teuthology.orchestra.run.trial093.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-25T14:20:58.368 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:20:58.368 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:20:58.369 INFO:teuthology.orchestra.run.trial090.stdout:(1/5): libcephsqlite-20.2.0-676.gc824a7fa.el9.x 315 kB/s | 164 kB 00:00 2026-02-25T14:20:58.406 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-test 2026-02-25T14:20:58.714 INFO:teuthology.orchestra.run.trial090.stdout:(2/5): libxslt-1.1.34-12.el9.x86_64.rpm 269 kB/s | 233 kB 00:00 2026-02-25T14:20:58.724 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:24 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:20:58.806 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:20:58.806 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:20:58.806 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repository Size 2026-02-25T14:20:58.806 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:20:58.807 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:20:58.807 INFO:teuthology.orchestra.run.trial093.stdout: ceph-test x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 84 M 2026-02-25T14:20:58.807 INFO:teuthology.orchestra.run.trial093.stdout:Installing dependencies: 2026-02-25T14:20:58.807 INFO:teuthology.orchestra.run.trial093.stdout: libcephsqlite x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 164 k 2026-02-25T14:20:58.807 INFO:teuthology.orchestra.run.trial093.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-25T14:20:58.807 INFO:teuthology.orchestra.run.trial093.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-25T14:20:58.807 INFO:teuthology.orchestra.run.trial093.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-25T14:20:58.807 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:20:58.807 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:20:58.807 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:20:58.807 INFO:teuthology.orchestra.run.trial093.stdout:Install 5 Packages 2026-02-25T14:20:58.807 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:20:58.807 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 85 M 2026-02-25T14:20:58.807 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 365 M 2026-02-25T14:20:58.807 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:20:58.871 INFO:teuthology.orchestra.run.trial090.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 405 kB/s | 64 kB 00:00 2026-02-25T14:20:59.161 INFO:teuthology.orchestra.run.trial090.stdout:(4/5): socat-1.7.4.1-8.el9.x86_64.rpm 383 kB/s | 303 kB 00:00 2026-02-25T14:20:59.311 INFO:teuthology.orchestra.run.trial093.stdout:(1/5): libcephsqlite-20.2.0-676.gc824a7fa.el9.x 371 kB/s | 164 kB 00:00 2026-02-25T14:20:59.733 INFO:teuthology.orchestra.run.trial093.stdout:(2/5): libxslt-1.1.34-12.el9.x86_64.rpm 269 kB/s | 233 kB 00:00 2026-02-25T14:20:59.891 INFO:teuthology.orchestra.run.trial093.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 404 kB/s | 64 kB 00:00 2026-02-25T14:21:00.023 INFO:teuthology.orchestra.run.trial093.stdout:(4/5): socat-1.7.4.1-8.el9.x86_64.rpm 426 kB/s | 303 kB 00:00 2026-02-25T14:21:02.148 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 4/5 2026-02-25T14:21:02.167 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 5/5 2026-02-25T14:21:02.500 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 5/5 2026-02-25T14:21:02.500 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 1/5 2026-02-25T14:21:02.500 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 2/5 2026-02-25T14:21:02.500 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-25T14:21:02.500 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-25T14:21:02.558 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-25T14:21:02.558 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:02.558 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:21:02.558 INFO:teuthology.orchestra.run.trial094.stdout: ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:02.558 INFO:teuthology.orchestra.run.trial094.stdout: libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:02.558 INFO:teuthology.orchestra.run.trial094.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-25T14:21:02.558 INFO:teuthology.orchestra.run.trial094.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-25T14:21:02.559 INFO:teuthology.orchestra.run.trial094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-25T14:21:02.559 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:02.559 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:02.594 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph 2026-02-25T14:21:02.905 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:28 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:02.992 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:02.995 INFO:teuthology.orchestra.run.trial094.stdout:====================================================================================== 2026-02-25T14:21:02.995 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repository Size 2026-02-25T14:21:02.995 INFO:teuthology.orchestra.run.trial094.stdout:====================================================================================== 2026-02-25T14:21:02.995 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:21:02.995 INFO:teuthology.orchestra.run.trial094.stdout: ceph x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 6.5 k 2026-02-25T14:21:02.995 INFO:teuthology.orchestra.run.trial094.stdout:Installing dependencies: 2026-02-25T14:21:02.996 INFO:teuthology.orchestra.run.trial094.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-25T14:21:02.996 INFO:teuthology.orchestra.run.trial094.stdout: ceph-grafana-dashboards noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 43 k 2026-02-25T14:21:02.996 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mds x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.3 M 2026-02-25T14:21:02.996 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 961 k 2026-02-25T14:21:02.996 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-modules-core noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 290 k 2026-02-25T14:21:02.996 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mon x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 5.0 M 2026-02-25T14:21:02.996 INFO:teuthology.orchestra.run.trial094.stdout: ceph-osd x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 17 M 2026-02-25T14:21:02.996 INFO:teuthology.orchestra.run.trial094.stdout: ceph-prometheus-alerts noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 17 k 2026-02-25T14:21:02.996 INFO:teuthology.orchestra.run.trial094.stdout: cephadm noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 1.0 M 2026-02-25T14:21:02.996 INFO:teuthology.orchestra.run.trial094.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-25T14:21:02.996 INFO:teuthology.orchestra.run.trial094.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-25T14:21:02.996 INFO:teuthology.orchestra.run.trial094.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-25T14:21:02.997 INFO:teuthology.orchestra.run.trial094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-25T14:21:02.997 INFO:teuthology.orchestra.run.trial094.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-25T14:21:02.997 INFO:teuthology.orchestra.run.trial094.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-25T14:21:02.997 INFO:teuthology.orchestra.run.trial094.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-25T14:21:02.997 INFO:teuthology.orchestra.run.trial094.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-25T14:21:02.997 INFO:teuthology.orchestra.run.trial094.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-25T14:21:02.997 INFO:teuthology.orchestra.run.trial094.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-25T14:21:02.997 INFO:teuthology.orchestra.run.trial094.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-25T14:21:02.997 INFO:teuthology.orchestra.run.trial094.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-25T14:21:02.997 INFO:teuthology.orchestra.run.trial094.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-25T14:21:02.997 INFO:teuthology.orchestra.run.trial094.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-25T14:21:02.997 INFO:teuthology.orchestra.run.trial094.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-25T14:21:02.997 INFO:teuthology.orchestra.run.trial094.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-25T14:21:02.998 INFO:teuthology.orchestra.run.trial094.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-25T14:21:02.998 INFO:teuthology.orchestra.run.trial094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-25T14:21:02.998 INFO:teuthology.orchestra.run.trial094.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-25T14:21:02.998 INFO:teuthology.orchestra.run.trial094.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-25T14:21:02.998 INFO:teuthology.orchestra.run.trial094.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-25T14:21:02.998 INFO:teuthology.orchestra.run.trial094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-25T14:21:02.998 INFO:teuthology.orchestra.run.trial094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-25T14:21:02.998 INFO:teuthology.orchestra.run.trial094.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-25T14:21:02.998 INFO:teuthology.orchestra.run.trial094.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-25T14:21:02.998 INFO:teuthology.orchestra.run.trial094.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-25T14:21:02.998 INFO:teuthology.orchestra.run.trial094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-25T14:21:02.998 INFO:teuthology.orchestra.run.trial094.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-25T14:21:02.998 INFO:teuthology.orchestra.run.trial094.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-25T14:21:02.999 INFO:teuthology.orchestra.run.trial094.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-25T14:21:02.999 INFO:teuthology.orchestra.run.trial094.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-25T14:21:02.999 INFO:teuthology.orchestra.run.trial094.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-25T14:21:02.999 INFO:teuthology.orchestra.run.trial094.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-25T14:21:02.999 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-25T14:21:02.999 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-25T14:21:02.999 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-25T14:21:02.999 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-25T14:21:02.999 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-25T14:21:02.999 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-25T14:21:02.999 INFO:teuthology.orchestra.run.trial094.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-25T14:21:02.999 INFO:teuthology.orchestra.run.trial094.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-25T14:21:02.999 INFO:teuthology.orchestra.run.trial094.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-25T14:21:03.000 INFO:teuthology.orchestra.run.trial094.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-25T14:21:03.000 INFO:teuthology.orchestra.run.trial094.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-25T14:21:03.000 INFO:teuthology.orchestra.run.trial094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-25T14:21:03.000 INFO:teuthology.orchestra.run.trial094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-25T14:21:03.000 INFO:teuthology.orchestra.run.trial094.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-25T14:21:03.000 INFO:teuthology.orchestra.run.trial094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-25T14:21:03.000 INFO:teuthology.orchestra.run.trial094.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-25T14:21:03.000 INFO:teuthology.orchestra.run.trial094.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-25T14:21:03.000 INFO:teuthology.orchestra.run.trial094.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-25T14:21:03.000 INFO:teuthology.orchestra.run.trial094.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-25T14:21:03.000 INFO:teuthology.orchestra.run.trial094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-25T14:21:03.000 INFO:teuthology.orchestra.run.trial094.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-25T14:21:03.001 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-25T14:21:03.001 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-25T14:21:03.001 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-25T14:21:03.001 INFO:teuthology.orchestra.run.trial094.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-25T14:21:03.001 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-25T14:21:03.001 INFO:teuthology.orchestra.run.trial094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-25T14:21:03.001 INFO:teuthology.orchestra.run.trial094.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-25T14:21:03.001 INFO:teuthology.orchestra.run.trial094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-25T14:21:03.001 INFO:teuthology.orchestra.run.trial094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-25T14:21:03.001 INFO:teuthology.orchestra.run.trial094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-25T14:21:03.001 INFO:teuthology.orchestra.run.trial094.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-25T14:21:03.001 INFO:teuthology.orchestra.run.trial094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-25T14:21:03.001 INFO:teuthology.orchestra.run.trial094.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-25T14:21:03.002 INFO:teuthology.orchestra.run.trial094.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-25T14:21:03.002 INFO:teuthology.orchestra.run.trial094.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-25T14:21:03.002 INFO:teuthology.orchestra.run.trial094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-25T14:21:03.002 INFO:teuthology.orchestra.run.trial094.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-25T14:21:03.002 INFO:teuthology.orchestra.run.trial094.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-25T14:21:03.002 INFO:teuthology.orchestra.run.trial094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-25T14:21:03.002 INFO:teuthology.orchestra.run.trial094.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-25T14:21:03.002 INFO:teuthology.orchestra.run.trial094.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-25T14:21:03.002 INFO:teuthology.orchestra.run.trial094.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-25T14:21:03.002 INFO:teuthology.orchestra.run.trial094.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-25T14:21:03.002 INFO:teuthology.orchestra.run.trial094.stdout:Installing weak dependencies: 2026-02-25T14:21:03.002 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-cephadm noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 173 k 2026-02-25T14:21:03.002 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-dashboard noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 11 M 2026-02-25T14:21:03.003 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-diskprediction-local noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 7.4 M 2026-02-25T14:21:03.003 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-k8sevents noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 22 k 2026-02-25T14:21:03.003 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-rook noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 50 k 2026-02-25T14:21:03.003 INFO:teuthology.orchestra.run.trial094.stdout: ceph-volume noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 297 k 2026-02-25T14:21:03.003 INFO:teuthology.orchestra.run.trial094.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-25T14:21:03.003 INFO:teuthology.orchestra.run.trial094.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-25T14:21:03.003 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:03.003 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:21:03.003 INFO:teuthology.orchestra.run.trial094.stdout:====================================================================================== 2026-02-25T14:21:03.003 INFO:teuthology.orchestra.run.trial094.stdout:Install 95 Packages 2026-02-25T14:21:03.003 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:03.004 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 101 M 2026-02-25T14:21:03.004 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 561 M 2026-02-25T14:21:03.005 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:21:03.785 INFO:teuthology.orchestra.run.trial094.stdout:(1/95): ceph-20.2.0-676.gc824a7fa.el9.x86_64.rp 23 kB/s | 6.5 kB 00:00 2026-02-25T14:21:04.370 INFO:teuthology.orchestra.run.trial094.stdout:(2/95): ceph-mds-20.2.0-676.gc824a7fa.el9.x86_6 2.7 MB/s | 2.3 MB 00:00 2026-02-25T14:21:04.635 INFO:teuthology.orchestra.run.trial094.stdout:(3/95): ceph-mgr-20.2.0-676.gc824a7fa.el9.x86_6 846 kB/s | 961 kB 00:01 2026-02-25T14:21:04.697 INFO:teuthology.orchestra.run.trial094.stdout:(4/95): ceph-grafana-dashboards-20.2.0-676.gc82 694 kB/s | 43 kB 00:00 2026-02-25T14:21:04.790 INFO:teuthology.orchestra.run.trial093.stdout:(5/5): ceph-test-20.2.0-676.gc824a7fa.el9.x86_6 14 MB/s | 84 MB 00:05 2026-02-25T14:21:04.793 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:04.793 INFO:teuthology.orchestra.run.trial093.stdout:Total 14 MB/s | 85 MB 00:05 2026-02-25T14:21:04.818 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:21:04.829 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:21:04.829 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:21:04.848 INFO:teuthology.orchestra.run.trial094.stdout:(5/95): ceph-mgr-cephadm-20.2.0-676.gc824a7fa.e 1.1 MB/s | 173 kB 00:00 2026-02-25T14:21:05.117 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:21:05.118 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:21:05.431 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:21:05.436 INFO:teuthology.orchestra.run.trial093.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-25T14:21:05.440 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-25T14:21:05.448 INFO:teuthology.orchestra.run.trial093.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-25T14:21:05.449 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 4/5 2026-02-25T14:21:05.656 INFO:teuthology.orchestra.run.trial094.stdout:(6/95): ceph-osd-20.2.0-676.gc824a7fa.el9.x86_6 13 MB/s | 17 MB 00:01 2026-02-25T14:21:06.173 INFO:teuthology.orchestra.run.trial094.stdout:(7/95): ceph-mgr-diskprediction-local-20.2.0-67 14 MB/s | 7.4 MB 00:00 2026-02-25T14:21:06.215 INFO:teuthology.orchestra.run.trial094.stdout:(8/95): ceph-mgr-k8sevents-20.2.0-676.gc824a7fa 516 kB/s | 22 kB 00:00 2026-02-25T14:21:06.265 INFO:teuthology.orchestra.run.trial094.stdout:(9/95): ceph-mgr-modules-core-20.2.0-676.gc824a 5.7 MB/s | 290 kB 00:00 2026-02-25T14:21:06.311 INFO:teuthology.orchestra.run.trial094.stdout:(10/95): ceph-mgr-rook-20.2.0-676.gc824a7fa.el9 1.1 MB/s | 50 kB 00:00 2026-02-25T14:21:06.358 INFO:teuthology.orchestra.run.trial094.stdout:(11/95): ceph-prometheus-alerts-20.2.0-676.gc82 373 kB/s | 17 kB 00:00 2026-02-25T14:21:06.410 INFO:teuthology.orchestra.run.trial094.stdout:(12/95): ceph-volume-20.2.0-676.gc824a7fa.el9.n 5.6 MB/s | 297 kB 00:00 2026-02-25T14:21:06.523 INFO:teuthology.orchestra.run.trial094.stdout:(13/95): cephadm-20.2.0-676.gc824a7fa.el9.noarc 8.9 MB/s | 1.0 MB 00:00 2026-02-25T14:21:06.633 INFO:teuthology.orchestra.run.trial094.stdout:(14/95): cryptsetup-2.8.1-3.el9.x86_64.rpm 3.1 MB/s | 351 kB 00:00 2026-02-25T14:21:06.655 INFO:teuthology.orchestra.run.trial094.stdout:(15/95): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.8 MB/s | 40 kB 00:00 2026-02-25T14:21:06.677 INFO:teuthology.orchestra.run.trial094.stdout:(16/95): libconfig-1.7.2-9.el9.x86_64.rpm 3.2 MB/s | 72 kB 00:00 2026-02-25T14:21:06.721 INFO:teuthology.orchestra.run.trial094.stdout:(17/95): libgfortran-11.5.0-14.el9.x86_64.rpm 18 MB/s | 794 kB 00:00 2026-02-25T14:21:06.746 INFO:teuthology.orchestra.run.trial094.stdout:(18/95): libquadmath-11.5.0-14.el9.x86_64.rpm 7.4 MB/s | 184 kB 00:00 2026-02-25T14:21:06.801 INFO:teuthology.orchestra.run.trial094.stdout:(19/95): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.5 MB/s | 253 kB 00:00 2026-02-25T14:21:06.848 INFO:teuthology.orchestra.run.trial094.stdout:(20/95): python3-cryptography-36.0.1-5.el9.x86_ 27 MB/s | 1.2 MB 00:00 2026-02-25T14:21:06.871 INFO:teuthology.orchestra.run.trial094.stdout:(21/95): python3-ply-3.11-14.el9.noarch.rpm 4.5 MB/s | 106 kB 00:00 2026-02-25T14:21:06.895 INFO:teuthology.orchestra.run.trial094.stdout:(22/95): python3-pycparser-2.20-6.el9.noarch.rp 5.6 MB/s | 135 kB 00:00 2026-02-25T14:21:06.919 INFO:teuthology.orchestra.run.trial094.stdout:(23/95): python3-pyparsing-2.4.7-9.el9.noarch.r 6.2 MB/s | 150 kB 00:00 2026-02-25T14:21:06.943 INFO:teuthology.orchestra.run.trial094.stdout:(24/95): python3-requests-2.25.1-10.el9.noarch. 5.2 MB/s | 126 kB 00:00 2026-02-25T14:21:06.967 INFO:teuthology.orchestra.run.trial094.stdout:(25/95): python3-urllib3-1.26.5-7.el9.noarch.rp 8.9 MB/s | 218 kB 00:00 2026-02-25T14:21:06.991 INFO:teuthology.orchestra.run.trial094.stdout:(26/95): unzip-6.0-59.el9.x86_64.rpm 7.3 MB/s | 182 kB 00:00 2026-02-25T14:21:07.015 INFO:teuthology.orchestra.run.trial094.stdout:(27/95): zip-3.0-35.el9.x86_64.rpm 11 MB/s | 266 kB 00:00 2026-02-25T14:21:07.085 INFO:teuthology.orchestra.run.trial094.stdout:(28/95): flexiblas-3.0.4-9.el9.x86_64.rpm 423 kB/s | 30 kB 00:00 2026-02-25T14:21:07.233 INFO:teuthology.orchestra.run.trial094.stdout:(29/95): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 20 MB/s | 3.0 MB 00:00 2026-02-25T14:21:07.253 INFO:teuthology.orchestra.run.trial094.stdout:(30/95): flexiblas-openblas-openmp-3.0.4-9.el9. 753 kB/s | 15 kB 00:00 2026-02-25T14:21:07.275 INFO:teuthology.orchestra.run.trial094.stdout:(31/95): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 11 MB/s | 246 kB 00:00 2026-02-25T14:21:07.296 INFO:teuthology.orchestra.run.trial094.stdout:(32/95): lua-5.4.4-4.el9.x86_64.rpm 8.6 MB/s | 188 kB 00:00 2026-02-25T14:21:07.316 INFO:teuthology.orchestra.run.trial094.stdout:(33/95): openblas-0.3.29-1.el9.x86_64.rpm 2.1 MB/s | 42 kB 00:00 2026-02-25T14:21:07.371 INFO:teuthology.orchestra.run.trial094.stdout:(34/95): openblas-openmp-0.3.29-1.el9.x86_64.rp 97 MB/s | 5.3 MB 00:00 2026-02-25T14:21:07.395 INFO:teuthology.orchestra.run.trial094.stdout:(35/95): protobuf-3.14.0-17.el9.x86_64.rpm 42 MB/s | 1.0 MB 00:00 2026-02-25T14:21:07.449 INFO:teuthology.orchestra.run.trial094.stdout:(36/95): python3-babel-2.9.1-2.el9.noarch.rpm 110 MB/s | 6.0 MB 00:00 2026-02-25T14:21:07.472 INFO:teuthology.orchestra.run.trial094.stdout:(37/95): python3-devel-3.9.25-3.el9.x86_64.rpm 11 MB/s | 244 kB 00:00 2026-02-25T14:21:07.494 INFO:teuthology.orchestra.run.trial094.stdout:(38/95): python3-jinja2-2.11.3-8.el9.noarch.rpm 11 MB/s | 249 kB 00:00 2026-02-25T14:21:07.514 INFO:teuthology.orchestra.run.trial094.stdout:(39/95): python3-jmespath-1.0.1-1.el9.noarch.rp 2.3 MB/s | 48 kB 00:00 2026-02-25T14:21:07.536 INFO:teuthology.orchestra.run.trial094.stdout:(40/95): python3-libstoragemgmt-1.10.1-1.el9.x8 8.0 MB/s | 177 kB 00:00 2026-02-25T14:21:07.561 INFO:teuthology.orchestra.run.trial094.stdout:(41/95): python3-lxml-4.6.5-3.el9.x86_64.rpm 49 MB/s | 1.2 MB 00:00 2026-02-25T14:21:07.582 INFO:teuthology.orchestra.run.trial094.stdout:(42/95): python3-markupsafe-1.1.1-12.el9.x86_64 1.7 MB/s | 35 kB 00:00 2026-02-25T14:21:07.635 INFO:teuthology.orchestra.run.trial094.stdout:(43/95): python3-numpy-1.23.5-2.el9.x86_64.rpm 117 MB/s | 6.1 MB 00:00 2026-02-25T14:21:07.658 INFO:teuthology.orchestra.run.trial094.stdout:(44/95): python3-numpy-f2py-1.23.5-2.el9.x86_64 19 MB/s | 442 kB 00:00 2026-02-25T14:21:07.679 INFO:teuthology.orchestra.run.trial094.stdout:(45/95): python3-packaging-20.9-5.el9.noarch.rp 3.6 MB/s | 77 kB 00:00 2026-02-25T14:21:07.700 INFO:teuthology.orchestra.run.trial094.stdout:(46/95): python3-protobuf-3.14.0-17.el9.noarch. 12 MB/s | 267 kB 00:00 2026-02-25T14:21:07.723 INFO:teuthology.orchestra.run.trial094.stdout:(47/95): python3-pyasn1-0.4.8-7.el9.noarch.rpm 7.0 MB/s | 157 kB 00:00 2026-02-25T14:21:07.745 INFO:teuthology.orchestra.run.trial094.stdout:(48/95): python3-pyasn1-modules-0.4.8-7.el9.noa 12 MB/s | 277 kB 00:00 2026-02-25T14:21:07.766 INFO:teuthology.orchestra.run.trial094.stdout:(49/95): python3-requests-oauthlib-1.3.0-12.el9 2.5 MB/s | 54 kB 00:00 2026-02-25T14:21:07.918 INFO:teuthology.orchestra.run.trial094.stdout:(50/95): python3-scipy-1.9.3-2.el9.x86_64.rpm 127 MB/s | 19 MB 00:00 2026-02-25T14:21:07.938 INFO:teuthology.orchestra.run.trial094.stdout:(51/95): python3-toml-0.10.2-6.el9.noarch.rpm 2.0 MB/s | 42 kB 00:00 2026-02-25T14:21:07.959 INFO:teuthology.orchestra.run.trial094.stdout:(52/95): xmlsec1-1.2.29-13.el9.x86_64.rpm 8.8 MB/s | 189 kB 00:00 2026-02-25T14:21:07.980 INFO:teuthology.orchestra.run.trial094.stdout:(53/95): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 4.3 MB/s | 90 kB 00:00 2026-02-25T14:21:08.032 INFO:teuthology.orchestra.run.trial094.stdout:(54/95): lua-devel-5.4.4-4.el9.x86_64.rpm 424 kB/s | 22 kB 00:00 2026-02-25T14:21:08.069 INFO:teuthology.orchestra.run.trial094.stdout:(55/95): protobuf-compiler-3.14.0-17.el9.x86_64 23 MB/s | 862 kB 00:00 2026-02-25T14:21:08.119 INFO:teuthology.orchestra.run.trial094.stdout:(56/95): abseil-cpp-20211102.0-4.el9.x86_64.rpm 11 MB/s | 551 kB 00:00 2026-02-25T14:21:08.126 INFO:teuthology.orchestra.run.trial094.stdout:(57/95): grpc-data-1.46.7-10.el9.noarch.rpm 2.7 MB/s | 19 kB 00:00 2026-02-25T14:21:08.134 INFO:teuthology.orchestra.run.trial094.stdout:(58/95): luarocks-3.9.2-5.el9.noarch.rpm 18 MB/s | 151 kB 00:00 2026-02-25T14:21:08.146 INFO:teuthology.orchestra.run.trial094.stdout:(59/95): python3-asyncssh-2.13.2-5.el9.noarch.r 48 MB/s | 548 kB 00:00 2026-02-25T14:21:08.155 INFO:teuthology.orchestra.run.trial094.stdout:(60/95): python3-autocommand-2.2.2-8.el9.noarch 3.6 MB/s | 29 kB 00:00 2026-02-25T14:21:08.163 INFO:teuthology.orchestra.run.trial094.stdout:(61/95): python3-backports-tarfile-1.2.0-1.el9. 7.4 MB/s | 60 kB 00:00 2026-02-25T14:21:08.170 INFO:teuthology.orchestra.run.trial094.stdout:(62/95): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 5.5 MB/s | 43 kB 00:00 2026-02-25T14:21:08.178 INFO:teuthology.orchestra.run.trial094.stdout:(63/95): python3-cachetools-4.2.4-1.el9.noarch. 4.2 MB/s | 32 kB 00:00 2026-02-25T14:21:08.185 INFO:teuthology.orchestra.run.trial094.stdout:(64/95): python3-certifi-2023.05.07-4.el9.noarc 1.9 MB/s | 14 kB 00:00 2026-02-25T14:21:08.195 INFO:teuthology.orchestra.run.trial094.stdout:(65/95): python3-cheroot-10.0.1-4.el9.noarch.rp 18 MB/s | 173 kB 00:00 2026-02-25T14:21:08.209 INFO:teuthology.orchestra.run.trial094.stdout:(66/95): python3-cherrypy-18.6.1-2.el9.noarch.r 25 MB/s | 358 kB 00:00 2026-02-25T14:21:08.219 INFO:teuthology.orchestra.run.trial094.stdout:(67/95): python3-google-auth-2.45.0-1.el9.noarc 25 MB/s | 254 kB 00:00 2026-02-25T14:21:08.271 INFO:teuthology.orchestra.run.trial094.stdout:(68/95): python3-grpcio-1.46.7-10.el9.x86_64.rp 39 MB/s | 2.0 MB 00:00 2026-02-25T14:21:08.280 INFO:teuthology.orchestra.run.trial094.stdout:(69/95): python3-grpcio-tools-1.46.7-10.el9.x86 15 MB/s | 144 kB 00:00 2026-02-25T14:21:08.290 INFO:teuthology.orchestra.run.trial094.stdout:(70/95): python3-influxdb-5.3.1-1.el9.noarch.rp 14 MB/s | 139 kB 00:00 2026-02-25T14:21:08.298 INFO:teuthology.orchestra.run.trial094.stdout:(71/95): python3-isodate-0.6.1-3.el9.noarch.rpm 6.9 MB/s | 56 kB 00:00 2026-02-25T14:21:08.305 INFO:teuthology.orchestra.run.trial094.stdout:(72/95): python3-jaraco-8.2.1-3.el9.noarch.rpm 1.5 MB/s | 11 kB 00:00 2026-02-25T14:21:08.313 INFO:teuthology.orchestra.run.trial094.stdout:(73/95): python3-jaraco-classes-3.2.1-5.el9.noa 2.4 MB/s | 18 kB 00:00 2026-02-25T14:21:08.320 INFO:teuthology.orchestra.run.trial094.stdout:(74/95): python3-jaraco-collections-3.0.0-8.el9 3.1 MB/s | 23 kB 00:00 2026-02-25T14:21:08.327 INFO:teuthology.orchestra.run.trial094.stdout:(75/95): python3-jaraco-context-6.0.1-3.el9.noa 2.6 MB/s | 20 kB 00:00 2026-02-25T14:21:08.335 INFO:teuthology.orchestra.run.trial094.stdout:(76/95): python3-jaraco-functools-3.5.0-2.el9.n 2.6 MB/s | 19 kB 00:00 2026-02-25T14:21:08.342 INFO:teuthology.orchestra.run.trial094.stdout:(77/95): python3-jaraco-text-4.0.0-2.el9.noarch 3.5 MB/s | 26 kB 00:00 2026-02-25T14:21:08.354 INFO:teuthology.orchestra.run.trial094.stdout:(78/95): python3-kubernetes-26.1.0-3.el9.noarch 88 MB/s | 1.0 MB 00:00 2026-02-25T14:21:08.363 INFO:teuthology.orchestra.run.trial094.stdout:(79/95): python3-more-itertools-8.12.0-2.el9.no 9.4 MB/s | 79 kB 00:00 2026-02-25T14:21:08.371 INFO:teuthology.orchestra.run.trial094.stdout:(80/95): python3-msgpack-1.0.3-2.el9.x86_64.rpm 11 MB/s | 86 kB 00:00 2026-02-25T14:21:08.379 INFO:teuthology.orchestra.run.trial094.stdout:(81/95): python3-natsort-7.1.1-5.el9.noarch.rpm 7.2 MB/s | 58 kB 00:00 2026-02-25T14:21:08.386 INFO:teuthology.orchestra.run.trial094.stdout:(82/95): python3-portend-3.1.0-2.el9.noarch.rpm 2.2 MB/s | 16 kB 00:00 2026-02-25T14:21:08.394 INFO:teuthology.orchestra.run.trial094.stdout:(83/95): python3-pyOpenSSL-21.0.0-1.el9.noarch. 11 MB/s | 90 kB 00:00 2026-02-25T14:21:08.401 INFO:teuthology.orchestra.run.trial094.stdout:(84/95): python3-repoze-lru-0.7-16.el9.noarch.r 4.0 MB/s | 31 kB 00:00 2026-02-25T14:21:08.410 INFO:teuthology.orchestra.run.trial094.stdout:(85/95): python3-routes-2.5.1-5.el9.noarch.rpm 21 MB/s | 188 kB 00:00 2026-02-25T14:21:08.418 INFO:teuthology.orchestra.run.trial094.stdout:(86/95): python3-rsa-4.9-2.el9.noarch.rpm 7.3 MB/s | 59 kB 00:00 2026-02-25T14:21:08.427 INFO:teuthology.orchestra.run.trial094.stdout:(87/95): python3-saml-1.16.0-1.el9.noarch.rpm 15 MB/s | 125 kB 00:00 2026-02-25T14:21:08.434 INFO:teuthology.orchestra.run.trial094.stdout:(88/95): python3-tempora-5.0.0-2.el9.noarch.rpm 4.6 MB/s | 36 kB 00:00 2026-02-25T14:21:08.443 INFO:teuthology.orchestra.run.trial094.stdout:(89/95): python3-typing-extensions-4.15.0-1.el9 10 MB/s | 86 kB 00:00 2026-02-25T14:21:08.450 INFO:teuthology.orchestra.run.trial094.stdout:(90/95): python3-websocket-client-1.2.3-2.el9.n 12 MB/s | 90 kB 00:00 2026-02-25T14:21:08.459 INFO:teuthology.orchestra.run.trial094.stdout:(91/95): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 5.9 MB/s | 48 kB 00:00 2026-02-25T14:21:08.466 INFO:teuthology.orchestra.run.trial094.stdout:(92/95): python3-xmltodict-0.12.0-15.el9.noarch 3.0 MB/s | 22 kB 00:00 2026-02-25T14:21:08.473 INFO:teuthology.orchestra.run.trial094.stdout:(93/95): python3-zc-lockfile-2.0-10.el9.noarch. 2.7 MB/s | 20 kB 00:00 2026-02-25T14:21:08.522 INFO:teuthology.orchestra.run.trial094.stdout:(94/95): ceph-mon-20.2.0-676.gc824a7fa.el9.x86_ 1.1 MB/s | 5.0 MB 00:04 2026-02-25T14:21:09.285 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 4/5 2026-02-25T14:21:09.303 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 5/5 2026-02-25T14:21:09.635 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 5/5 2026-02-25T14:21:09.635 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 1/5 2026-02-25T14:21:09.635 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 2/5 2026-02-25T14:21:09.636 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-25T14:21:09.636 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-25T14:21:09.694 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-25T14:21:09.694 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:09.694 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:21:09.694 INFO:teuthology.orchestra.run.trial093.stdout: ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:09.694 INFO:teuthology.orchestra.run.trial093.stdout: libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:09.694 INFO:teuthology.orchestra.run.trial093.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-25T14:21:09.694 INFO:teuthology.orchestra.run.trial093.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-25T14:21:09.695 INFO:teuthology.orchestra.run.trial093.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-25T14:21:09.695 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:09.695 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:09.733 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph 2026-02-25T14:21:10.053 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:36 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:10.140 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:10.143 INFO:teuthology.orchestra.run.trial093.stdout:====================================================================================== 2026-02-25T14:21:10.144 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repository Size 2026-02-25T14:21:10.144 INFO:teuthology.orchestra.run.trial093.stdout:====================================================================================== 2026-02-25T14:21:10.144 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:21:10.144 INFO:teuthology.orchestra.run.trial093.stdout: ceph x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 6.5 k 2026-02-25T14:21:10.144 INFO:teuthology.orchestra.run.trial093.stdout:Installing dependencies: 2026-02-25T14:21:10.144 INFO:teuthology.orchestra.run.trial093.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-25T14:21:10.144 INFO:teuthology.orchestra.run.trial093.stdout: ceph-grafana-dashboards noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 43 k 2026-02-25T14:21:10.144 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mds x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.3 M 2026-02-25T14:21:10.144 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 961 k 2026-02-25T14:21:10.145 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-modules-core noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 290 k 2026-02-25T14:21:10.145 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mon x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 5.0 M 2026-02-25T14:21:10.145 INFO:teuthology.orchestra.run.trial093.stdout: ceph-osd x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 17 M 2026-02-25T14:21:10.145 INFO:teuthology.orchestra.run.trial093.stdout: ceph-prometheus-alerts noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 17 k 2026-02-25T14:21:10.145 INFO:teuthology.orchestra.run.trial093.stdout: cephadm noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 1.0 M 2026-02-25T14:21:10.145 INFO:teuthology.orchestra.run.trial093.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-25T14:21:10.145 INFO:teuthology.orchestra.run.trial093.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-25T14:21:10.145 INFO:teuthology.orchestra.run.trial093.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-25T14:21:10.145 INFO:teuthology.orchestra.run.trial093.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-25T14:21:10.145 INFO:teuthology.orchestra.run.trial093.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-25T14:21:10.146 INFO:teuthology.orchestra.run.trial093.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-25T14:21:10.146 INFO:teuthology.orchestra.run.trial093.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-25T14:21:10.146 INFO:teuthology.orchestra.run.trial093.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-25T14:21:10.146 INFO:teuthology.orchestra.run.trial093.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-25T14:21:10.146 INFO:teuthology.orchestra.run.trial093.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-25T14:21:10.146 INFO:teuthology.orchestra.run.trial093.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-25T14:21:10.146 INFO:teuthology.orchestra.run.trial093.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-25T14:21:10.146 INFO:teuthology.orchestra.run.trial093.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-25T14:21:10.146 INFO:teuthology.orchestra.run.trial093.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-25T14:21:10.146 INFO:teuthology.orchestra.run.trial093.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-25T14:21:10.147 INFO:teuthology.orchestra.run.trial093.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-25T14:21:10.147 INFO:teuthology.orchestra.run.trial093.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-25T14:21:10.147 INFO:teuthology.orchestra.run.trial093.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-25T14:21:10.147 INFO:teuthology.orchestra.run.trial093.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-25T14:21:10.147 INFO:teuthology.orchestra.run.trial093.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-25T14:21:10.147 INFO:teuthology.orchestra.run.trial093.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-25T14:21:10.147 INFO:teuthology.orchestra.run.trial093.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-25T14:21:10.147 INFO:teuthology.orchestra.run.trial093.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-25T14:21:10.147 INFO:teuthology.orchestra.run.trial093.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-25T14:21:10.147 INFO:teuthology.orchestra.run.trial093.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-25T14:21:10.147 INFO:teuthology.orchestra.run.trial093.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-25T14:21:10.148 INFO:teuthology.orchestra.run.trial093.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-25T14:21:10.148 INFO:teuthology.orchestra.run.trial093.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-25T14:21:10.148 INFO:teuthology.orchestra.run.trial093.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-25T14:21:10.148 INFO:teuthology.orchestra.run.trial093.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-25T14:21:10.148 INFO:teuthology.orchestra.run.trial093.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-25T14:21:10.148 INFO:teuthology.orchestra.run.trial093.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-25T14:21:10.148 INFO:teuthology.orchestra.run.trial093.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-25T14:21:10.148 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-25T14:21:10.148 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-25T14:21:10.148 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-25T14:21:10.148 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-25T14:21:10.148 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-25T14:21:10.149 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-25T14:21:10.149 INFO:teuthology.orchestra.run.trial093.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-25T14:21:10.149 INFO:teuthology.orchestra.run.trial093.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-25T14:21:10.149 INFO:teuthology.orchestra.run.trial093.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-25T14:21:10.149 INFO:teuthology.orchestra.run.trial093.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-25T14:21:10.149 INFO:teuthology.orchestra.run.trial093.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-25T14:21:10.149 INFO:teuthology.orchestra.run.trial093.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-25T14:21:10.149 INFO:teuthology.orchestra.run.trial093.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-25T14:21:10.149 INFO:teuthology.orchestra.run.trial093.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-25T14:21:10.149 INFO:teuthology.orchestra.run.trial093.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-25T14:21:10.150 INFO:teuthology.orchestra.run.trial093.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-25T14:21:10.150 INFO:teuthology.orchestra.run.trial093.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-25T14:21:10.150 INFO:teuthology.orchestra.run.trial093.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-25T14:21:10.150 INFO:teuthology.orchestra.run.trial093.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-25T14:21:10.150 INFO:teuthology.orchestra.run.trial093.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-25T14:21:10.150 INFO:teuthology.orchestra.run.trial093.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-25T14:21:10.150 INFO:teuthology.orchestra.run.trial093.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-25T14:21:10.150 INFO:teuthology.orchestra.run.trial093.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-25T14:21:10.150 INFO:teuthology.orchestra.run.trial093.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-25T14:21:10.150 INFO:teuthology.orchestra.run.trial093.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-25T14:21:10.150 INFO:teuthology.orchestra.run.trial093.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-25T14:21:10.151 INFO:teuthology.orchestra.run.trial093.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-25T14:21:10.151 INFO:teuthology.orchestra.run.trial093.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-25T14:21:10.151 INFO:teuthology.orchestra.run.trial093.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-25T14:21:10.151 INFO:teuthology.orchestra.run.trial093.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-25T14:21:10.151 INFO:teuthology.orchestra.run.trial093.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-25T14:21:10.151 INFO:teuthology.orchestra.run.trial093.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-25T14:21:10.151 INFO:teuthology.orchestra.run.trial093.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-25T14:21:10.151 INFO:teuthology.orchestra.run.trial093.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-25T14:21:10.151 INFO:teuthology.orchestra.run.trial093.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-25T14:21:10.151 INFO:teuthology.orchestra.run.trial093.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-25T14:21:10.151 INFO:teuthology.orchestra.run.trial093.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-25T14:21:10.152 INFO:teuthology.orchestra.run.trial093.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-25T14:21:10.152 INFO:teuthology.orchestra.run.trial093.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-25T14:21:10.152 INFO:teuthology.orchestra.run.trial093.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-25T14:21:10.152 INFO:teuthology.orchestra.run.trial093.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-25T14:21:10.152 INFO:teuthology.orchestra.run.trial093.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-25T14:21:10.152 INFO:teuthology.orchestra.run.trial093.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-25T14:21:10.152 INFO:teuthology.orchestra.run.trial093.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-25T14:21:10.152 INFO:teuthology.orchestra.run.trial093.stdout:Installing weak dependencies: 2026-02-25T14:21:10.152 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-cephadm noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 173 k 2026-02-25T14:21:10.152 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-dashboard noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 11 M 2026-02-25T14:21:10.152 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-diskprediction-local noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 7.4 M 2026-02-25T14:21:10.153 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-k8sevents noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 22 k 2026-02-25T14:21:10.153 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-rook noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 50 k 2026-02-25T14:21:10.153 INFO:teuthology.orchestra.run.trial093.stdout: ceph-volume noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 297 k 2026-02-25T14:21:10.153 INFO:teuthology.orchestra.run.trial093.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-25T14:21:10.153 INFO:teuthology.orchestra.run.trial093.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-25T14:21:10.153 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:10.153 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:21:10.153 INFO:teuthology.orchestra.run.trial093.stdout:====================================================================================== 2026-02-25T14:21:10.153 INFO:teuthology.orchestra.run.trial093.stdout:Install 95 Packages 2026-02-25T14:21:10.153 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:10.154 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 101 M 2026-02-25T14:21:10.155 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 561 M 2026-02-25T14:21:10.155 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:21:11.533 INFO:teuthology.orchestra.run.trial093.stdout:(1/95): ceph-20.2.0-676.gc824a7fa.el9.x86_64.rp 19 kB/s | 6.5 kB 00:00 2026-02-25T14:21:11.887 INFO:teuthology.orchestra.run.trial093.stdout:(2/95): ceph-mgr-20.2.0-676.gc824a7fa.el9.x86_6 1.4 MB/s | 961 kB 00:00 2026-02-25T14:21:12.022 INFO:teuthology.orchestra.run.trial094.stdout:(95/95): ceph-mgr-dashboard-20.2.0-676.gc824a7f 1.5 MB/s | 11 MB 00:07 2026-02-25T14:21:12.024 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:12.025 INFO:teuthology.orchestra.run.trial094.stdout:Total 11 MB/s | 101 MB 00:09 2026-02-25T14:21:12.227 INFO:teuthology.orchestra.run.trial093.stdout:(3/95): ceph-mds-20.2.0-676.gc824a7fa.el9.x86_6 2.3 MB/s | 2.3 MB 00:01 2026-02-25T14:21:12.290 INFO:teuthology.orchestra.run.trial093.stdout:(4/95): ceph-grafana-dashboards-20.2.0-676.gc82 679 kB/s | 43 kB 00:00 2026-02-25T14:21:12.409 INFO:teuthology.orchestra.run.trial093.stdout:(5/95): ceph-mgr-cephadm-20.2.0-676.gc824a7fa.e 1.4 MB/s | 173 kB 00:00 2026-02-25T14:21:12.439 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:21:12.475 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:21:12.475 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:21:12.907 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:21:12.907 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:21:13.373 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:21:13.375 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/95 2026-02-25T14:21:13.379 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/95 2026-02-25T14:21:13.401 INFO:teuthology.orchestra.run.trial094.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/95 2026-02-25T14:21:13.436 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 4/95 2026-02-25T14:21:13.445 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/95 2026-02-25T14:21:13.451 INFO:teuthology.orchestra.run.trial094.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 6/95 2026-02-25T14:21:13.453 INFO:teuthology.orchestra.run.trial094.stdout: Installing : lua-5.4.4-4.el9.x86_64 7/95 2026-02-25T14:21:13.457 INFO:teuthology.orchestra.run.trial094.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 8/95 2026-02-25T14:21:13.481 INFO:teuthology.orchestra.run.trial094.stdout: Installing : unzip-6.0-59.el9.x86_64 9/95 2026-02-25T14:21:13.494 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 10/95 2026-02-25T14:21:13.497 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 11/95 2026-02-25T14:21:13.503 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 12/95 2026-02-25T14:21:13.505 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 13/95 2026-02-25T14:21:13.516 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 14/95 2026-02-25T14:21:13.522 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 15/95 2026-02-25T14:21:13.547 INFO:teuthology.orchestra.run.trial094.stdout: Installing : zip-3.0-35.el9.x86_64 16/95 2026-02-25T14:21:13.550 INFO:teuthology.orchestra.run.trial094.stdout: Installing : luarocks-3.9.2-5.el9.noarch 17/95 2026-02-25T14:21:13.558 INFO:teuthology.orchestra.run.trial094.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 18/95 2026-02-25T14:21:13.609 INFO:teuthology.orchestra.run.trial094.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 19/95 2026-02-25T14:21:13.622 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 20/95 2026-02-25T14:21:13.626 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 21/95 2026-02-25T14:21:13.630 INFO:teuthology.orchestra.run.trial094.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 22/95 2026-02-25T14:21:13.636 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 23/95 2026-02-25T14:21:13.640 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 24/95 2026-02-25T14:21:13.643 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 25/95 2026-02-25T14:21:13.658 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 26/95 2026-02-25T14:21:13.662 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 27/95 2026-02-25T14:21:13.668 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 28/95 2026-02-25T14:21:13.679 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 29/95 2026-02-25T14:21:13.688 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 30/95 2026-02-25T14:21:13.694 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 31/95 2026-02-25T14:21:13.714 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 32/95 2026-02-25T14:21:13.722 INFO:teuthology.orchestra.run.trial093.stdout:(6/95): ceph-osd-20.2.0-676.gc824a7fa.el9.x86_6 9.3 MB/s | 17 MB 00:01 2026-02-25T14:21:13.725 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 33/95 2026-02-25T14:21:13.741 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 34/95 2026-02-25T14:21:13.746 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 35/95 2026-02-25T14:21:13.754 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 36/95 2026-02-25T14:21:13.794 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 37/95 2026-02-25T14:21:14.123 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 38/95 2026-02-25T14:21:14.133 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 39/95 2026-02-25T14:21:14.137 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 40/95 2026-02-25T14:21:14.143 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 41/95 2026-02-25T14:21:14.144 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 42/95 2026-02-25T14:21:14.172 INFO:teuthology.orchestra.run.trial094.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 43/95 2026-02-25T14:21:14.216 INFO:teuthology.orchestra.run.trial094.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 44/95 2026-02-25T14:21:14.223 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 45/95 2026-02-25T14:21:14.227 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 46/95 2026-02-25T14:21:14.232 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 47/95 2026-02-25T14:21:14.236 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 48/95 2026-02-25T14:21:14.243 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 49/95 2026-02-25T14:21:14.246 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 50/95 2026-02-25T14:21:14.275 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 51/95 2026-02-25T14:21:14.285 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 52/95 2026-02-25T14:21:14.290 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 53/95 2026-02-25T14:21:14.297 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 54/95 2026-02-25T14:21:14.333 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 55/95 2026-02-25T14:21:14.560 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 56/95 2026-02-25T14:21:14.585 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 57/95 2026-02-25T14:21:14.588 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 58/95 2026-02-25T14:21:14.648 INFO:teuthology.orchestra.run.trial094.stdout: Installing : openblas-0.3.29-1.el9.x86_64 59/95 2026-02-25T14:21:14.650 INFO:teuthology.orchestra.run.trial094.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 60/95 2026-02-25T14:21:14.671 INFO:teuthology.orchestra.run.trial094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 61/95 2026-02-25T14:21:14.749 INFO:teuthology.orchestra.run.trial093.stdout:(7/95): ceph-mon-20.2.0-676.gc824a7fa.el9.x86_6 1.6 MB/s | 5.0 MB 00:03 2026-02-25T14:21:14.814 INFO:teuthology.orchestra.run.trial093.stdout:(8/95): ceph-mgr-k8sevents-20.2.0-676.gc824a7fa 341 kB/s | 22 kB 00:00 2026-02-25T14:21:14.997 INFO:teuthology.orchestra.run.trial093.stdout:(9/95): ceph-mgr-modules-core-20.2.0-676.gc824a 1.6 MB/s | 290 kB 00:00 2026-02-25T14:21:15.000 INFO:teuthology.orchestra.run.trial094.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 62/95 2026-02-25T14:21:15.075 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 63/95 2026-02-25T14:21:15.208 INFO:teuthology.orchestra.run.trial093.stdout:(10/95): ceph-mgr-dashboard-20.2.0-676.gc824a7f 3.8 MB/s | 11 MB 00:02 2026-02-25T14:21:15.223 INFO:teuthology.orchestra.run.trial093.stdout:(11/95): ceph-mgr-rook-20.2.0-676.gc824a7fa.el9 221 kB/s | 50 kB 00:00 2026-02-25T14:21:15.270 INFO:teuthology.orchestra.run.trial093.stdout:(12/95): ceph-prometheus-alerts-20.2.0-676.gc82 281 kB/s | 17 kB 00:00 2026-02-25T14:21:15.292 INFO:teuthology.orchestra.run.trial093.stdout:(13/95): ceph-mgr-diskprediction-local-20.2.0-6 4.7 MB/s | 7.4 MB 00:01 2026-02-25T14:21:15.358 INFO:teuthology.orchestra.run.trial093.stdout:(14/95): ceph-volume-20.2.0-676.gc824a7fa.el9.n 2.2 MB/s | 297 kB 00:00 2026-02-25T14:21:15.404 INFO:teuthology.orchestra.run.trial093.stdout:(15/95): cryptsetup-2.8.1-3.el9.x86_64.rpm 3.1 MB/s | 351 kB 00:00 2026-02-25T14:21:15.420 INFO:teuthology.orchestra.run.trial093.stdout:(16/95): ledmon-libs-1.1.0-3.el9.x86_64.rpm 647 kB/s | 40 kB 00:00 2026-02-25T14:21:15.426 INFO:teuthology.orchestra.run.trial093.stdout:(17/95): libconfig-1.7.2-9.el9.x86_64.rpm 3.2 MB/s | 72 kB 00:00 2026-02-25T14:21:15.449 INFO:teuthology.orchestra.run.trial093.stdout:(18/95): libquadmath-11.5.0-14.el9.x86_64.rpm 8.0 MB/s | 184 kB 00:00 2026-02-25T14:21:15.475 INFO:teuthology.orchestra.run.trial093.stdout:(19/95): python3-cffi-1.14.5-5.el9.x86_64.rpm 9.3 MB/s | 253 kB 00:00 2026-02-25T14:21:15.502 INFO:teuthology.orchestra.run.trial093.stdout:(20/95): cephadm-20.2.0-676.gc824a7fa.el9.noarc 4.3 MB/s | 1.0 MB 00:00 2026-02-25T14:21:15.506 INFO:teuthology.orchestra.run.trial093.stdout:(21/95): libgfortran-11.5.0-14.el9.x86_64.rpm 9.1 MB/s | 794 kB 00:00 2026-02-25T14:21:15.523 INFO:teuthology.orchestra.run.trial093.stdout:(22/95): python3-cryptography-36.0.1-5.el9.x86_ 26 MB/s | 1.2 MB 00:00 2026-02-25T14:21:15.530 INFO:teuthology.orchestra.run.trial093.stdout:(23/95): python3-pycparser-2.20-6.el9.noarch.rp 5.6 MB/s | 135 kB 00:00 2026-02-25T14:21:15.547 INFO:teuthology.orchestra.run.trial093.stdout:(24/95): python3-pyparsing-2.4.7-9.el9.noarch.r 6.2 MB/s | 150 kB 00:00 2026-02-25T14:21:15.552 INFO:teuthology.orchestra.run.trial093.stdout:(25/95): python3-requests-2.25.1-10.el9.noarch. 5.7 MB/s | 126 kB 00:00 2026-02-25T14:21:15.566 INFO:teuthology.orchestra.run.trial093.stdout:(26/95): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2026-02-25T14:21:15.571 INFO:teuthology.orchestra.run.trial093.stdout:(27/95): python3-urllib3-1.26.5-7.el9.noarch.rp 8.9 MB/s | 218 kB 00:00 2026-02-25T14:21:15.575 INFO:teuthology.orchestra.run.trial093.stdout:(28/95): unzip-6.0-59.el9.x86_64.rpm 7.6 MB/s | 182 kB 00:00 2026-02-25T14:21:15.610 INFO:teuthology.orchestra.run.trial093.stdout:(29/95): zip-3.0-35.el9.x86_64.rpm 5.9 MB/s | 266 kB 00:00 2026-02-25T14:21:15.629 INFO:teuthology.orchestra.run.trial093.stdout:(30/95): flexiblas-3.0.4-9.el9.x86_64.rpm 506 kB/s | 30 kB 00:00 2026-02-25T14:21:15.673 INFO:teuthology.orchestra.run.trial093.stdout:(31/95): flexiblas-openblas-openmp-3.0.4-9.el9. 236 kB/s | 15 kB 00:00 2026-02-25T14:21:15.689 INFO:teuthology.orchestra.run.trial093.stdout:(32/95): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 4.0 MB/s | 246 kB 00:00 2026-02-25T14:21:15.710 INFO:teuthology.orchestra.run.trial093.stdout:(33/95): openblas-0.3.29-1.el9.x86_64.rpm 2.0 MB/s | 42 kB 00:00 2026-02-25T14:21:15.737 INFO:teuthology.orchestra.run.trial093.stdout:(34/95): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 188 kB 00:00 2026-02-25T14:21:15.754 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 64/95 2026-02-25T14:21:15.770 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 65/95 2026-02-25T14:21:15.775 INFO:teuthology.orchestra.run.trial093.stdout:(35/95): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 15 MB/s | 3.0 MB 00:00 2026-02-25T14:21:15.781 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 66/95 2026-02-25T14:21:15.787 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/95 2026-02-25T14:21:15.801 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-ply-3.11-14.el9.noarch 68/95 2026-02-25T14:21:15.815 INFO:teuthology.orchestra.run.trial093.stdout:(36/95): protobuf-3.14.0-17.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2026-02-25T14:21:15.817 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 69/95 2026-02-25T14:21:15.837 INFO:teuthology.orchestra.run.trial093.stdout:(37/95): python3-babel-2.9.1-2.el9.noarch.rpm 98 MB/s | 6.0 MB 00:00 2026-02-25T14:21:15.840 INFO:teuthology.orchestra.run.trial093.stdout:(38/95): python3-devel-3.9.25-3.el9.x86_64.rpm 9.6 MB/s | 244 kB 00:00 2026-02-25T14:21:15.863 INFO:teuthology.orchestra.run.trial093.stdout:(39/95): python3-jinja2-2.11.3-8.el9.noarch.rpm 9.4 MB/s | 249 kB 00:00 2026-02-25T14:21:15.864 INFO:teuthology.orchestra.run.trial093.stdout:(40/95): python3-jmespath-1.0.1-1.el9.noarch.rp 1.9 MB/s | 48 kB 00:00 2026-02-25T14:21:15.888 INFO:teuthology.orchestra.run.trial093.stdout:(41/95): python3-libstoragemgmt-1.10.1-1.el9.x8 6.8 MB/s | 177 kB 00:00 2026-02-25T14:21:15.893 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 70/95 2026-02-25T14:21:15.894 INFO:teuthology.orchestra.run.trial093.stdout:(42/95): openblas-openmp-0.3.29-1.el9.x86_64.rp 29 MB/s | 5.3 MB 00:00 2026-02-25T14:21:15.902 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 71/95 2026-02-25T14:21:15.909 INFO:teuthology.orchestra.run.trial093.stdout:(43/95): python3-markupsafe-1.1.1-12.el9.x86_64 1.6 MB/s | 35 kB 00:00 2026-02-25T14:21:15.912 INFO:teuthology.orchestra.run.trial093.stdout:(44/95): python3-lxml-4.6.5-3.el9.x86_64.rpm 25 MB/s | 1.2 MB 00:00 2026-02-25T14:21:15.926 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 72/95 2026-02-25T14:21:15.933 INFO:teuthology.orchestra.run.trial093.stdout:(45/95): python3-numpy-f2py-1.23.5-2.el9.x86_64 18 MB/s | 442 kB 00:00 2026-02-25T14:21:15.956 INFO:teuthology.orchestra.run.trial093.stdout:(46/95): python3-numpy-1.23.5-2.el9.x86_64.rpm 100 MB/s | 6.1 MB 00:00 2026-02-25T14:21:15.956 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 73/95 2026-02-25T14:21:15.957 INFO:teuthology.orchestra.run.trial093.stdout:(47/95): python3-packaging-20.9-5.el9.noarch.rp 1.7 MB/s | 77 kB 00:00 2026-02-25T14:21:15.959 INFO:teuthology.orchestra.run.trial093.stdout:(48/95): python3-protobuf-3.14.0-17.el9.noarch. 10 MB/s | 267 kB 00:00 2026-02-25T14:21:15.980 INFO:teuthology.orchestra.run.trial093.stdout:(49/95): python3-requests-oauthlib-1.3.0-12.el9 2.5 MB/s | 54 kB 00:00 2026-02-25T14:21:15.981 INFO:teuthology.orchestra.run.trial093.stdout:(50/95): python3-pyasn1-0.4.8-7.el9.noarch.rpm 6.0 MB/s | 157 kB 00:00 2026-02-25T14:21:15.984 INFO:teuthology.orchestra.run.trial093.stdout:(51/95): python3-pyasn1-modules-0.4.8-7.el9.noa 10 MB/s | 277 kB 00:00 2026-02-25T14:21:16.005 INFO:teuthology.orchestra.run.trial093.stdout:(52/95): python3-toml-0.10.2-6.el9.noarch.rpm 1.7 MB/s | 42 kB 00:00 2026-02-25T14:21:16.007 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 74/95 2026-02-25T14:21:16.010 INFO:teuthology.orchestra.run.trial093.stdout:(53/95): xmlsec1-1.2.29-13.el9.x86_64.rpm 7.0 MB/s | 189 kB 00:00 2026-02-25T14:21:16.013 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 75/95 2026-02-25T14:21:16.017 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/95 2026-02-25T14:21:16.018 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 77/95 2026-02-25T14:21:16.035 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 78/95 2026-02-25T14:21:16.035 INFO:teuthology.orchestra.run.trial094.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-25T14:21:16.035 INFO:teuthology.orchestra.run.trial094.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-25T14:21:16.035 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:16.042 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 78/95 2026-02-25T14:21:16.055 INFO:teuthology.orchestra.run.trial093.stdout:(54/95): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 1.8 MB/s | 90 kB 00:00 2026-02-25T14:21:16.066 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 78/95 2026-02-25T14:21:16.066 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-25T14:21:16.066 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:16.101 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 79/95 2026-02-25T14:21:16.157 INFO:teuthology.orchestra.run.trial093.stdout:(55/95): python3-scipy-1.9.3-2.el9.x86_64.rpm 109 MB/s | 19 MB 00:00 2026-02-25T14:21:16.164 INFO:teuthology.orchestra.run.trial094.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 80/95 2026-02-25T14:21:16.167 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 81/95 2026-02-25T14:21:16.179 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 81/95 2026-02-25T14:21:16.179 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:21:16.179 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-25T14:21:16.179 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:16.255 INFO:teuthology.orchestra.run.trial093.stdout:(56/95): lua-devel-5.4.4-4.el9.x86_64.rpm 91 kB/s | 22 kB 00:00 2026-02-25T14:21:16.364 INFO:teuthology.orchestra.run.trial093.stdout:(57/95): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2026-02-25T14:21:16.378 INFO:teuthology.orchestra.run.trial093.stdout:(58/95): abseil-cpp-20211102.0-4.el9.x86_64.rpm 2.4 MB/s | 551 kB 00:00 2026-02-25T14:21:16.422 INFO:teuthology.orchestra.run.trial093.stdout:(59/95): python3-asyncssh-2.13.2-5.el9.noarch.r 12 MB/s | 548 kB 00:00 2026-02-25T14:21:16.448 INFO:teuthology.orchestra.run.trial093.stdout:(60/95): python3-autocommand-2.2.2-8.el9.noarch 1.1 MB/s | 29 kB 00:00 2026-02-25T14:21:16.453 INFO:teuthology.orchestra.run.trial093.stdout:(61/95): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2026-02-25T14:21:16.476 INFO:teuthology.orchestra.run.trial093.stdout:(62/95): python3-backports-tarfile-1.2.0-1.el9. 2.1 MB/s | 60 kB 00:00 2026-02-25T14:21:16.483 INFO:teuthology.orchestra.run.trial093.stdout:(63/95): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.4 MB/s | 43 kB 00:00 2026-02-25T14:21:16.502 INFO:teuthology.orchestra.run.trial093.stdout:(64/95): python3-cachetools-4.2.4-1.el9.noarch. 1.2 MB/s | 32 kB 00:00 2026-02-25T14:21:16.510 INFO:teuthology.orchestra.run.trial093.stdout:(65/95): python3-certifi-2023.05.07-4.el9.noarc 539 kB/s | 14 kB 00:00 2026-02-25T14:21:16.532 INFO:teuthology.orchestra.run.trial093.stdout:(66/95): python3-cheroot-10.0.1-4.el9.noarch.rp 5.6 MB/s | 173 kB 00:00 2026-02-25T14:21:16.565 INFO:teuthology.orchestra.run.trial093.stdout:(67/95): python3-google-auth-2.45.0-1.el9.noarc 7.6 MB/s | 254 kB 00:00 2026-02-25T14:21:16.572 INFO:teuthology.orchestra.run.trial093.stdout:(68/95): python3-cherrypy-18.6.1-2.el9.noarch.r 5.6 MB/s | 358 kB 00:00 2026-02-25T14:21:16.603 INFO:teuthology.orchestra.run.trial093.stdout:(69/95): python3-grpcio-tools-1.46.7-10.el9.x86 4.6 MB/s | 144 kB 00:00 2026-02-25T14:21:16.633 INFO:teuthology.orchestra.run.trial093.stdout:(70/95): python3-influxdb-5.3.1-1.el9.noarch.rp 4.5 MB/s | 139 kB 00:00 2026-02-25T14:21:16.645 INFO:teuthology.orchestra.run.trial093.stdout:(71/95): python3-grpcio-1.46.7-10.el9.x86_64.rp 25 MB/s | 2.0 MB 00:00 2026-02-25T14:21:16.661 INFO:teuthology.orchestra.run.trial093.stdout:(72/95): python3-isodate-0.6.1-3.el9.noarch.rpm 2.0 MB/s | 56 kB 00:00 2026-02-25T14:21:16.671 INFO:teuthology.orchestra.run.trial093.stdout:(73/95): python3-jaraco-8.2.1-3.el9.noarch.rpm 412 kB/s | 11 kB 00:00 2026-02-25T14:21:16.687 INFO:teuthology.orchestra.run.trial093.stdout:(74/95): python3-jaraco-classes-3.2.1-5.el9.noa 677 kB/s | 18 kB 00:00 2026-02-25T14:21:16.697 INFO:teuthology.orchestra.run.trial093.stdout:(75/95): python3-jaraco-collections-3.0.0-8.el9 889 kB/s | 23 kB 00:00 2026-02-25T14:21:16.710 INFO:teuthology.orchestra.run.trial093.stdout:(76/95): protobuf-compiler-3.14.0-17.el9.x86_64 1.3 MB/s | 862 kB 00:00 2026-02-25T14:21:16.714 INFO:teuthology.orchestra.run.trial093.stdout:(77/95): python3-jaraco-context-6.0.1-3.el9.noa 735 kB/s | 20 kB 00:00 2026-02-25T14:21:16.723 INFO:teuthology.orchestra.run.trial093.stdout:(78/95): python3-jaraco-functools-3.5.0-2.el9.n 748 kB/s | 19 kB 00:00 2026-02-25T14:21:16.750 INFO:teuthology.orchestra.run.trial093.stdout:(79/95): python3-more-itertools-8.12.0-2.el9.no 2.9 MB/s | 79 kB 00:00 2026-02-25T14:21:16.777 INFO:teuthology.orchestra.run.trial093.stdout:(80/95): python3-msgpack-1.0.3-2.el9.x86_64.rpm 3.1 MB/s | 86 kB 00:00 2026-02-25T14:21:16.778 INFO:teuthology.orchestra.run.trial093.stdout:(81/95): python3-kubernetes-26.1.0-3.el9.noarch 16 MB/s | 1.0 MB 00:00 2026-02-25T14:21:16.803 INFO:teuthology.orchestra.run.trial093.stdout:(82/95): python3-natsort-7.1.1-5.el9.noarch.rpm 2.1 MB/s | 58 kB 00:00 2026-02-25T14:21:16.804 INFO:teuthology.orchestra.run.trial093.stdout:(83/95): python3-portend-3.1.0-2.el9.noarch.rpm 638 kB/s | 16 kB 00:00 2026-02-25T14:21:16.809 INFO:teuthology.orchestra.run.trial093.stdout:(84/95): python3-jaraco-text-4.0.0-2.el9.noarch 268 kB/s | 26 kB 00:00 2026-02-25T14:21:16.831 INFO:teuthology.orchestra.run.trial093.stdout:(85/95): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.2 MB/s | 90 kB 00:00 2026-02-25T14:21:16.831 INFO:teuthology.orchestra.run.trial093.stdout:(86/95): python3-repoze-lru-0.7-16.el9.noarch.r 1.1 MB/s | 31 kB 00:00 2026-02-25T14:21:16.858 INFO:teuthology.orchestra.run.trial093.stdout:(87/95): python3-rsa-4.9-2.el9.noarch.rpm 2.2 MB/s | 59 kB 00:00 2026-02-25T14:21:16.859 INFO:teuthology.orchestra.run.trial093.stdout:(88/95): python3-saml-1.16.0-1.el9.noarch.rpm 4.5 MB/s | 125 kB 00:00 2026-02-25T14:21:16.863 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 82/95 2026-02-25T14:21:16.884 INFO:teuthology.orchestra.run.trial093.stdout:(89/95): python3-tempora-5.0.0-2.el9.noarch.rpm 1.3 MB/s | 36 kB 00:00 2026-02-25T14:21:16.886 INFO:teuthology.orchestra.run.trial093.stdout:(90/95): python3-typing-extensions-4.15.0-1.el9 3.1 MB/s | 86 kB 00:00 2026-02-25T14:21:16.888 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 82/95 2026-02-25T14:21:16.888 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:21:16.888 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-25T14:21:16.888 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T14:21:16.888 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T14:21:16.888 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:16.891 INFO:teuthology.orchestra.run.trial093.stdout:(91/95): python3-routes-2.5.1-5.el9.noarch.rpm 2.2 MB/s | 188 kB 00:00 2026-02-25T14:21:16.911 INFO:teuthology.orchestra.run.trial093.stdout:(92/95): python3-websocket-client-1.2.3-2.el9.n 3.2 MB/s | 90 kB 00:00 2026-02-25T14:21:16.912 INFO:teuthology.orchestra.run.trial093.stdout:(93/95): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 1.8 MB/s | 48 kB 00:00 2026-02-25T14:21:16.915 INFO:teuthology.orchestra.run.trial093.stdout:(94/95): python3-xmltodict-0.12.0-15.el9.noarch 944 kB/s | 22 kB 00:00 2026-02-25T14:21:16.938 INFO:teuthology.orchestra.run.trial093.stdout:(95/95): python3-zc-lockfile-2.0-10.el9.noarch. 761 kB/s | 20 kB 00:00 2026-02-25T14:21:16.941 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:16.941 INFO:teuthology.orchestra.run.trial093.stdout:Total 15 MB/s | 101 MB 00:06 2026-02-25T14:21:16.946 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 83/95 2026-02-25T14:21:16.948 INFO:teuthology.orchestra.run.trial094.stdout: Installing : cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 83/95 2026-02-25T14:21:16.955 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-prometheus-alerts-2:20.2.0-676.gc824a7fa.el9. 84/95 2026-02-25T14:21:16.979 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-grafana-dashboards-2:20.2.0-676.gc824a7fa.el9 85/95 2026-02-25T14:21:16.980 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 86/95 2026-02-25T14:21:17.402 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:21:17.439 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:21:17.439 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:21:17.870 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:21:17.871 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:21:18.137 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 86/95 2026-02-25T14:21:18.142 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noar 87/95 2026-02-25T14:21:18.337 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:21:18.339 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/95 2026-02-25T14:21:18.343 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/95 2026-02-25T14:21:18.364 INFO:teuthology.orchestra.run.trial093.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/95 2026-02-25T14:21:18.396 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 4/95 2026-02-25T14:21:18.406 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/95 2026-02-25T14:21:18.411 INFO:teuthology.orchestra.run.trial093.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 6/95 2026-02-25T14:21:18.413 INFO:teuthology.orchestra.run.trial093.stdout: Installing : lua-5.4.4-4.el9.x86_64 7/95 2026-02-25T14:21:18.417 INFO:teuthology.orchestra.run.trial093.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 8/95 2026-02-25T14:21:18.440 INFO:teuthology.orchestra.run.trial093.stdout: Installing : unzip-6.0-59.el9.x86_64 9/95 2026-02-25T14:21:18.453 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 10/95 2026-02-25T14:21:18.456 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 11/95 2026-02-25T14:21:18.462 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 12/95 2026-02-25T14:21:18.464 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 13/95 2026-02-25T14:21:18.475 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 14/95 2026-02-25T14:21:18.481 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 15/95 2026-02-25T14:21:18.504 INFO:teuthology.orchestra.run.trial093.stdout: Installing : zip-3.0-35.el9.x86_64 16/95 2026-02-25T14:21:18.507 INFO:teuthology.orchestra.run.trial093.stdout: Installing : luarocks-3.9.2-5.el9.noarch 17/95 2026-02-25T14:21:18.515 INFO:teuthology.orchestra.run.trial093.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 18/95 2026-02-25T14:21:18.564 INFO:teuthology.orchestra.run.trial093.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 19/95 2026-02-25T14:21:18.577 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 20/95 2026-02-25T14:21:18.579 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-rsa-4.9-2.el9.noarch 21/95 2026-02-25T14:21:18.584 INFO:teuthology.orchestra.run.trial093.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 22/95 2026-02-25T14:21:18.589 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 23/95 2026-02-25T14:21:18.593 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 24/95 2026-02-25T14:21:18.596 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 25/95 2026-02-25T14:21:18.600 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noar 87/95 2026-02-25T14:21:18.601 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7 88/95 2026-02-25T14:21:18.611 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 26/95 2026-02-25T14:21:18.615 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7 88/95 2026-02-25T14:21:18.615 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 27/95 2026-02-25T14:21:18.616 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noar 89/95 2026-02-25T14:21:18.620 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 28/95 2026-02-25T14:21:18.631 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 29/95 2026-02-25T14:21:18.640 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 30/95 2026-02-25T14:21:18.646 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 31/95 2026-02-25T14:21:18.665 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 32/95 2026-02-25T14:21:18.672 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noar 89/95 2026-02-25T14:21:18.676 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 33/95 2026-02-25T14:21:18.691 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 34/95 2026-02-25T14:21:18.696 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 35/95 2026-02-25T14:21:18.703 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 36/95 2026-02-25T14:21:18.716 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mgr-modules-core-2:20.2.0-676.gc824a7fa.el9.n 90/95 2026-02-25T14:21:18.718 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 91/95 2026-02-25T14:21:18.739 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 91/95 2026-02-25T14:21:18.739 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:21:18.739 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-25T14:21:18.739 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T14:21:18.739 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T14:21:18.739 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:18.743 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 37/95 2026-02-25T14:21:18.750 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 92/95 2026-02-25T14:21:18.982 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 92/95 2026-02-25T14:21:18.984 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 93/95 2026-02-25T14:21:19.005 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 93/95 2026-02-25T14:21:19.005 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:21:19.005 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-25T14:21:19.006 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T14:21:19.006 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T14:21:19.006 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:19.061 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 38/95 2026-02-25T14:21:19.071 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 39/95 2026-02-25T14:21:19.075 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 40/95 2026-02-25T14:21:19.080 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 41/95 2026-02-25T14:21:19.082 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 42/95 2026-02-25T14:21:19.105 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 94/95 2026-02-25T14:21:19.109 INFO:teuthology.orchestra.run.trial093.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 43/95 2026-02-25T14:21:19.126 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 94/95 2026-02-25T14:21:19.126 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:21:19.126 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-25T14:21:19.126 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T14:21:19.126 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T14:21:19.126 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:19.145 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 95/95 2026-02-25T14:21:19.151 INFO:teuthology.orchestra.run.trial093.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 44/95 2026-02-25T14:21:19.158 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 45/95 2026-02-25T14:21:19.162 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 46/95 2026-02-25T14:21:19.167 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 47/95 2026-02-25T14:21:19.171 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 48/95 2026-02-25T14:21:19.177 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 49/95 2026-02-25T14:21:19.181 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 50/95 2026-02-25T14:21:19.208 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 51/95 2026-02-25T14:21:19.218 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 52/95 2026-02-25T14:21:19.223 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 53/95 2026-02-25T14:21:19.230 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 54/95 2026-02-25T14:21:19.264 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 55/95 2026-02-25T14:21:19.484 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 56/95 2026-02-25T14:21:19.507 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 57/95 2026-02-25T14:21:19.511 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 58/95 2026-02-25T14:21:19.569 INFO:teuthology.orchestra.run.trial093.stdout: Installing : openblas-0.3.29-1.el9.x86_64 59/95 2026-02-25T14:21:19.571 INFO:teuthology.orchestra.run.trial093.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 60/95 2026-02-25T14:21:19.592 INFO:teuthology.orchestra.run.trial093.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 61/95 2026-02-25T14:21:19.833 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 95/95 2026-02-25T14:21:19.834 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 1/95 2026-02-25T14:21:19.834 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 2/95 2026-02-25T14:21:19.834 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 3/95 2026-02-25T14:21:19.834 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 4/95 2026-02-25T14:21:19.834 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 5/95 2026-02-25T14:21:19.834 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-grafana-dashboards-2:20.2.0-676.gc824a7fa.el9 6/95 2026-02-25T14:21:19.834 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 7/95 2026-02-25T14:21:19.834 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noar 8/95 2026-02-25T14:21:19.834 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7 9/95 2026-02-25T14:21:19.834 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noar 10/95 2026-02-25T14:21:19.835 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mgr-modules-core-2:20.2.0-676.gc824a7fa.el9.n 11/95 2026-02-25T14:21:19.835 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 12/95 2026-02-25T14:21:19.835 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-prometheus-alerts-2:20.2.0-676.gc824a7fa.el9. 13/95 2026-02-25T14:21:19.835 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 14/95 2026-02-25T14:21:19.835 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 15/95 2026-02-25T14:21:19.835 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 16/95 2026-02-25T14:21:19.835 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/95 2026-02-25T14:21:19.835 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/95 2026-02-25T14:21:19.835 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/95 2026-02-25T14:21:19.835 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/95 2026-02-25T14:21:19.835 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/95 2026-02-25T14:21:19.835 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/95 2026-02-25T14:21:19.836 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/95 2026-02-25T14:21:19.836 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/95 2026-02-25T14:21:19.836 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/95 2026-02-25T14:21:19.836 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/95 2026-02-25T14:21:19.836 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/95 2026-02-25T14:21:19.836 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/95 2026-02-25T14:21:19.836 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : zip-3.0-35.el9.x86_64 29/95 2026-02-25T14:21:19.836 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/95 2026-02-25T14:21:19.836 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/95 2026-02-25T14:21:19.836 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/95 2026-02-25T14:21:19.836 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/95 2026-02-25T14:21:19.837 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/95 2026-02-25T14:21:19.837 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/95 2026-02-25T14:21:19.837 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/95 2026-02-25T14:21:19.837 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/95 2026-02-25T14:21:19.837 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/95 2026-02-25T14:21:19.837 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 39/95 2026-02-25T14:21:19.837 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/95 2026-02-25T14:21:19.837 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 41/95 2026-02-25T14:21:19.837 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 42/95 2026-02-25T14:21:19.838 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 43/95 2026-02-25T14:21:19.838 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/95 2026-02-25T14:21:19.838 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/95 2026-02-25T14:21:19.838 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/95 2026-02-25T14:21:19.838 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/95 2026-02-25T14:21:19.838 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/95 2026-02-25T14:21:19.838 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 49/95 2026-02-25T14:21:19.838 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 50/95 2026-02-25T14:21:19.838 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/95 2026-02-25T14:21:19.838 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/95 2026-02-25T14:21:19.838 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/95 2026-02-25T14:21:19.838 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/95 2026-02-25T14:21:19.838 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/95 2026-02-25T14:21:19.839 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/95 2026-02-25T14:21:19.839 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/95 2026-02-25T14:21:19.839 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/95 2026-02-25T14:21:19.839 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/95 2026-02-25T14:21:19.839 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/95 2026-02-25T14:21:19.839 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/95 2026-02-25T14:21:19.839 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/95 2026-02-25T14:21:19.839 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/95 2026-02-25T14:21:19.839 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/95 2026-02-25T14:21:19.839 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/95 2026-02-25T14:21:19.839 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/95 2026-02-25T14:21:19.839 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/95 2026-02-25T14:21:19.839 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/95 2026-02-25T14:21:19.840 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/95 2026-02-25T14:21:19.840 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/95 2026-02-25T14:21:19.840 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/95 2026-02-25T14:21:19.840 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/95 2026-02-25T14:21:19.840 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/95 2026-02-25T14:21:19.840 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/95 2026-02-25T14:21:19.840 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/95 2026-02-25T14:21:19.840 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/95 2026-02-25T14:21:19.840 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/95 2026-02-25T14:21:19.840 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/95 2026-02-25T14:21:19.840 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/95 2026-02-25T14:21:19.840 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/95 2026-02-25T14:21:19.840 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 81/95 2026-02-25T14:21:19.841 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 82/95 2026-02-25T14:21:19.841 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 83/95 2026-02-25T14:21:19.841 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 84/95 2026-02-25T14:21:19.841 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 85/95 2026-02-25T14:21:19.841 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 86/95 2026-02-25T14:21:19.841 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 87/95 2026-02-25T14:21:19.841 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 88/95 2026-02-25T14:21:19.841 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 89/95 2026-02-25T14:21:19.842 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 90/95 2026-02-25T14:21:19.842 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 91/95 2026-02-25T14:21:19.842 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 92/95 2026-02-25T14:21:19.842 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 93/95 2026-02-25T14:21:19.842 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 94/95 2026-02-25T14:21:19.901 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 95/95 2026-02-25T14:21:19.901 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:19.901 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:21:19.901 INFO:teuthology.orchestra.run.trial094.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-25T14:21:19.902 INFO:teuthology.orchestra.run.trial094.stdout: ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:19.902 INFO:teuthology.orchestra.run.trial094.stdout: ceph-grafana-dashboards-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:19.902 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:19.902 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:19.902 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:19.902 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:19.902 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:19.902 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:19.902 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-modules-core-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:19.902 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:19.902 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:19.902 INFO:teuthology.orchestra.run.trial094.stdout: ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:19.903 INFO:teuthology.orchestra.run.trial094.stdout: ceph-prometheus-alerts-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:19.903 INFO:teuthology.orchestra.run.trial094.stdout: ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:19.903 INFO:teuthology.orchestra.run.trial094.stdout: cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:19.903 INFO:teuthology.orchestra.run.trial094.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-25T14:21:19.903 INFO:teuthology.orchestra.run.trial094.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-25T14:21:19.903 INFO:teuthology.orchestra.run.trial094.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-25T14:21:19.903 INFO:teuthology.orchestra.run.trial094.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-25T14:21:19.903 INFO:teuthology.orchestra.run.trial094.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-25T14:21:19.903 INFO:teuthology.orchestra.run.trial094.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-25T14:21:19.903 INFO:teuthology.orchestra.run.trial094.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-25T14:21:19.903 INFO:teuthology.orchestra.run.trial094.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-25T14:21:19.904 INFO:teuthology.orchestra.run.trial094.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-25T14:21:19.904 INFO:teuthology.orchestra.run.trial094.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T14:21:19.904 INFO:teuthology.orchestra.run.trial094.stdout: lua-5.4.4-4.el9.x86_64 2026-02-25T14:21:19.904 INFO:teuthology.orchestra.run.trial094.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-25T14:21:19.904 INFO:teuthology.orchestra.run.trial094.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-25T14:21:19.904 INFO:teuthology.orchestra.run.trial094.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-25T14:21:19.904 INFO:teuthology.orchestra.run.trial094.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-25T14:21:19.905 INFO:teuthology.orchestra.run.trial094.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-25T14:21:19.905 INFO:teuthology.orchestra.run.trial094.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-25T14:21:19.905 INFO:teuthology.orchestra.run.trial094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-25T14:21:19.905 INFO:teuthology.orchestra.run.trial094.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-25T14:21:19.905 INFO:teuthology.orchestra.run.trial094.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-25T14:21:19.905 INFO:teuthology.orchestra.run.trial094.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-25T14:21:19.906 INFO:teuthology.orchestra.run.trial094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-25T14:21:19.906 INFO:teuthology.orchestra.run.trial094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-25T14:21:19.906 INFO:teuthology.orchestra.run.trial094.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-25T14:21:19.906 INFO:teuthology.orchestra.run.trial094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-25T14:21:19.906 INFO:teuthology.orchestra.run.trial094.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-25T14:21:19.906 INFO:teuthology.orchestra.run.trial094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-25T14:21:19.906 INFO:teuthology.orchestra.run.trial094.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-25T14:21:19.907 INFO:teuthology.orchestra.run.trial094.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-25T14:21:19.907 INFO:teuthology.orchestra.run.trial094.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-25T14:21:19.907 INFO:teuthology.orchestra.run.trial094.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-25T14:21:19.907 INFO:teuthology.orchestra.run.trial094.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-25T14:21:19.907 INFO:teuthology.orchestra.run.trial094.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-25T14:21:19.907 INFO:teuthology.orchestra.run.trial094.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-25T14:21:19.907 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-25T14:21:19.907 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-25T14:21:19.907 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-25T14:21:19.907 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-25T14:21:19.907 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-25T14:21:19.908 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-25T14:21:19.908 INFO:teuthology.orchestra.run.trial094.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-25T14:21:19.908 INFO:teuthology.orchestra.run.trial094.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-25T14:21:19.908 INFO:teuthology.orchestra.run.trial094.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-25T14:21:19.908 INFO:teuthology.orchestra.run.trial094.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T14:21:19.908 INFO:teuthology.orchestra.run.trial094.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-25T14:21:19.908 INFO:teuthology.orchestra.run.trial094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-25T14:21:19.908 INFO:teuthology.orchestra.run.trial094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-25T14:21:19.908 INFO:teuthology.orchestra.run.trial094.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-25T14:21:19.908 INFO:teuthology.orchestra.run.trial094.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-25T14:21:19.908 INFO:teuthology.orchestra.run.trial094.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-25T14:21:19.908 INFO:teuthology.orchestra.run.trial094.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-25T14:21:19.908 INFO:teuthology.orchestra.run.trial094.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-25T14:21:19.909 INFO:teuthology.orchestra.run.trial094.stdout: python3-ply-3.11-14.el9.noarch 2026-02-25T14:21:19.909 INFO:teuthology.orchestra.run.trial094.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-25T14:21:19.909 INFO:teuthology.orchestra.run.trial094.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-25T14:21:19.909 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-25T14:21:19.909 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-25T14:21:19.909 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-25T14:21:19.909 INFO:teuthology.orchestra.run.trial094.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-25T14:21:19.909 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-25T14:21:19.909 INFO:teuthology.orchestra.run.trial094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-25T14:21:19.909 INFO:teuthology.orchestra.run.trial094.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-25T14:21:19.909 INFO:teuthology.orchestra.run.trial094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-25T14:21:19.909 INFO:teuthology.orchestra.run.trial094.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-25T14:21:19.909 INFO:teuthology.orchestra.run.trial094.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-25T14:21:19.910 INFO:teuthology.orchestra.run.trial094.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-25T14:21:19.910 INFO:teuthology.orchestra.run.trial094.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-25T14:21:19.910 INFO:teuthology.orchestra.run.trial094.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-25T14:21:19.910 INFO:teuthology.orchestra.run.trial094.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-25T14:21:19.910 INFO:teuthology.orchestra.run.trial094.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-25T14:21:19.910 INFO:teuthology.orchestra.run.trial094.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-25T14:21:19.910 INFO:teuthology.orchestra.run.trial094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-25T14:21:19.910 INFO:teuthology.orchestra.run.trial094.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-25T14:21:19.910 INFO:teuthology.orchestra.run.trial094.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-25T14:21:19.910 INFO:teuthology.orchestra.run.trial094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-25T14:21:19.910 INFO:teuthology.orchestra.run.trial094.stdout: unzip-6.0-59.el9.x86_64 2026-02-25T14:21:19.910 INFO:teuthology.orchestra.run.trial094.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-25T14:21:19.910 INFO:teuthology.orchestra.run.trial094.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-25T14:21:19.911 INFO:teuthology.orchestra.run.trial094.stdout: zip-3.0-35.el9.x86_64 2026-02-25T14:21:19.911 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:19.911 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:19.912 INFO:teuthology.orchestra.run.trial093.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 62/95 2026-02-25T14:21:19.955 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-base 2026-02-25T14:21:19.984 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 63/95 2026-02-25T14:21:20.267 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:46 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:20.332 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:20.350 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:20.351 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:21:20.351 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:20.372 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install cephadm 2026-02-25T14:21:20.653 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 64/95 2026-02-25T14:21:20.670 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 65/95 2026-02-25T14:21:20.680 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 66/95 2026-02-25T14:21:20.683 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:46 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:20.686 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/95 2026-02-25T14:21:20.700 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-ply-3.11-14.el9.noarch 68/95 2026-02-25T14:21:20.715 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 69/95 2026-02-25T14:21:20.747 INFO:teuthology.orchestra.run.trial094.stdout:Package cephadm-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:21:20.765 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:20.766 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:21:20.766 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:20.786 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T14:21:20.789 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 70/95 2026-02-25T14:21:20.797 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 71/95 2026-02-25T14:21:20.820 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 72/95 2026-02-25T14:21:20.849 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 73/95 2026-02-25T14:21:20.899 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 74/95 2026-02-25T14:21:20.905 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 75/95 2026-02-25T14:21:20.908 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/95 2026-02-25T14:21:20.910 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 77/95 2026-02-25T14:21:20.928 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 78/95 2026-02-25T14:21:20.928 INFO:teuthology.orchestra.run.trial093.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-25T14:21:20.929 INFO:teuthology.orchestra.run.trial093.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-25T14:21:20.929 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:20.937 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 78/95 2026-02-25T14:21:20.964 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 78/95 2026-02-25T14:21:20.965 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-25T14:21:20.965 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:21.000 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 79/95 2026-02-25T14:21:21.063 INFO:teuthology.orchestra.run.trial093.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 80/95 2026-02-25T14:21:21.065 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 81/95 2026-02-25T14:21:21.080 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 81/95 2026-02-25T14:21:21.080 INFO:teuthology.orchestra.run.trial093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:21:21.080 INFO:teuthology.orchestra.run.trial093.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-25T14:21:21.080 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:21.105 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:47 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:21.188 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:21.189 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:21.189 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:21:21.190 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:21.190 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:21:21.190 INFO:teuthology.orchestra.run.trial094.stdout: ceph-immutable-object-cache x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 154 k 2026-02-25T14:21:21.190 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:21.190 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:21:21.190 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:21.190 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:21:21.190 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:21.190 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 154 k 2026-02-25T14:21:21.190 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 447 k 2026-02-25T14:21:21.191 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:21:21.572 INFO:teuthology.orchestra.run.trial094.stdout:ceph-immutable-object-cache-20.2.0-676.gc824a7f 402 kB/s | 154 kB 00:00 2026-02-25T14:21:21.572 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:21.573 INFO:teuthology.orchestra.run.trial094.stdout:Total 401 kB/s | 154 kB 00:00 2026-02-25T14:21:21.573 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:21:21.578 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:21:21.578 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:21:21.610 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:21:21.610 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:21:21.666 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:21:21.668 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.e 1/1 2026-02-25T14:21:21.689 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.e 1/1 2026-02-25T14:21:21.689 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:21:21.689 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-25T14:21:21.689 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:21.752 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 82/95 2026-02-25T14:21:21.779 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 82/95 2026-02-25T14:21:21.779 INFO:teuthology.orchestra.run.trial093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:21:21.780 INFO:teuthology.orchestra.run.trial093.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-25T14:21:21.780 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T14:21:21.780 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T14:21:21.780 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:21.839 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 83/95 2026-02-25T14:21:21.841 INFO:teuthology.orchestra.run.trial093.stdout: Installing : cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 83/95 2026-02-25T14:21:21.848 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-prometheus-alerts-2:20.2.0-676.gc824a7fa.el9. 84/95 2026-02-25T14:21:21.871 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-grafana-dashboards-2:20.2.0-676.gc824a7fa.el9 85/95 2026-02-25T14:21:21.872 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 86/95 2026-02-25T14:21:21.983 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.e 1/1 2026-02-25T14:21:21.983 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:21.983 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:21:21.983 INFO:teuthology.orchestra.run.trial094.stdout: ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:21.983 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:21.983 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:22.010 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr 2026-02-25T14:21:22.320 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:48 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:22.387 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:22.405 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:22.406 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:21:22.406 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:22.426 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T14:21:22.743 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:48 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:22.808 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:21:22.826 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:22.827 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:21:22.827 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:22.848 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T14:21:23.011 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 86/95 2026-02-25T14:21:23.016 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noar 87/95 2026-02-25T14:21:23.166 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:49 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:23.231 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:21:23.250 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:23.250 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:21:23.250 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:23.272 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr-rook 2026-02-25T14:21:23.470 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noar 87/95 2026-02-25T14:21:23.471 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7 88/95 2026-02-25T14:21:23.484 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7 88/95 2026-02-25T14:21:23.484 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noar 89/95 2026-02-25T14:21:23.541 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noar 89/95 2026-02-25T14:21:23.584 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-mgr-modules-core-2:20.2.0-676.gc824a7fa.el9.n 90/95 2026-02-25T14:21:23.585 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 91/95 2026-02-25T14:21:23.591 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:49 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:23.607 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 91/95 2026-02-25T14:21:23.607 INFO:teuthology.orchestra.run.trial093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:21:23.607 INFO:teuthology.orchestra.run.trial093.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-25T14:21:23.607 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T14:21:23.607 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T14:21:23.607 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:23.618 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 92/95 2026-02-25T14:21:23.656 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:21:23.675 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:23.675 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:21:23.676 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:23.697 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T14:21:23.842 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 92/95 2026-02-25T14:21:23.843 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 93/95 2026-02-25T14:21:23.865 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 93/95 2026-02-25T14:21:23.865 INFO:teuthology.orchestra.run.trial093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:21:23.865 INFO:teuthology.orchestra.run.trial093.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-25T14:21:23.865 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T14:21:23.865 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T14:21:23.865 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:23.962 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 94/95 2026-02-25T14:21:23.984 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 94/95 2026-02-25T14:21:23.984 INFO:teuthology.orchestra.run.trial093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:21:23.984 INFO:teuthology.orchestra.run.trial093.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-25T14:21:23.984 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T14:21:23.984 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T14:21:23.984 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:24.002 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 95/95 2026-02-25T14:21:24.012 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:50 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:24.077 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:21:24.096 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:24.097 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:21:24.097 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:24.118 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-fuse 2026-02-25T14:21:24.432 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:50 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:24.516 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:24.516 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:24.516 INFO:teuthology.orchestra.run.trial094.stdout: Package Architecture Version Repository Size 2026-02-25T14:21:24.516 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:24.516 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:21:24.516 INFO:teuthology.orchestra.run.trial094.stdout: ceph-fuse x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 942 k 2026-02-25T14:21:24.516 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:24.516 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:21:24.517 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:24.517 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:21:24.517 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:24.517 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 942 k 2026-02-25T14:21:24.517 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 2.7 M 2026-02-25T14:21:24.517 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:21:24.701 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 95/95 2026-02-25T14:21:24.701 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 1/95 2026-02-25T14:21:24.701 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 2/95 2026-02-25T14:21:24.701 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 3/95 2026-02-25T14:21:24.701 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 4/95 2026-02-25T14:21:24.701 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 5/95 2026-02-25T14:21:24.702 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-grafana-dashboards-2:20.2.0-676.gc824a7fa.el9 6/95 2026-02-25T14:21:24.702 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 7/95 2026-02-25T14:21:24.702 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noar 8/95 2026-02-25T14:21:24.702 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7 9/95 2026-02-25T14:21:24.702 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noar 10/95 2026-02-25T14:21:24.702 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-mgr-modules-core-2:20.2.0-676.gc824a7fa.el9.n 11/95 2026-02-25T14:21:24.702 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 12/95 2026-02-25T14:21:24.702 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-prometheus-alerts-2:20.2.0-676.gc824a7fa.el9. 13/95 2026-02-25T14:21:24.702 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 14/95 2026-02-25T14:21:24.702 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 15/95 2026-02-25T14:21:24.702 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 16/95 2026-02-25T14:21:24.702 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/95 2026-02-25T14:21:24.702 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/95 2026-02-25T14:21:24.703 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/95 2026-02-25T14:21:24.703 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/95 2026-02-25T14:21:24.703 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/95 2026-02-25T14:21:24.703 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/95 2026-02-25T14:21:24.703 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/95 2026-02-25T14:21:24.703 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/95 2026-02-25T14:21:24.703 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/95 2026-02-25T14:21:24.703 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/95 2026-02-25T14:21:24.703 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/95 2026-02-25T14:21:24.703 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/95 2026-02-25T14:21:24.703 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : zip-3.0-35.el9.x86_64 29/95 2026-02-25T14:21:24.703 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/95 2026-02-25T14:21:24.704 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/95 2026-02-25T14:21:24.704 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/95 2026-02-25T14:21:24.704 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/95 2026-02-25T14:21:24.704 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/95 2026-02-25T14:21:24.704 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/95 2026-02-25T14:21:24.704 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/95 2026-02-25T14:21:24.704 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/95 2026-02-25T14:21:24.704 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/95 2026-02-25T14:21:24.704 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 39/95 2026-02-25T14:21:24.704 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/95 2026-02-25T14:21:24.704 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 41/95 2026-02-25T14:21:24.704 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 42/95 2026-02-25T14:21:24.705 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 43/95 2026-02-25T14:21:24.705 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/95 2026-02-25T14:21:24.705 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/95 2026-02-25T14:21:24.705 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/95 2026-02-25T14:21:24.705 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/95 2026-02-25T14:21:24.705 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/95 2026-02-25T14:21:24.705 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 49/95 2026-02-25T14:21:24.705 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 50/95 2026-02-25T14:21:24.705 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/95 2026-02-25T14:21:24.705 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/95 2026-02-25T14:21:24.705 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/95 2026-02-25T14:21:24.705 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/95 2026-02-25T14:21:24.705 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/95 2026-02-25T14:21:24.706 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/95 2026-02-25T14:21:24.706 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/95 2026-02-25T14:21:24.706 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/95 2026-02-25T14:21:24.706 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/95 2026-02-25T14:21:24.706 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/95 2026-02-25T14:21:24.706 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/95 2026-02-25T14:21:24.706 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/95 2026-02-25T14:21:24.706 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/95 2026-02-25T14:21:24.706 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/95 2026-02-25T14:21:24.706 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/95 2026-02-25T14:21:24.706 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/95 2026-02-25T14:21:24.706 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/95 2026-02-25T14:21:24.706 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/95 2026-02-25T14:21:24.707 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/95 2026-02-25T14:21:24.707 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/95 2026-02-25T14:21:24.707 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/95 2026-02-25T14:21:24.707 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/95 2026-02-25T14:21:24.707 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/95 2026-02-25T14:21:24.707 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/95 2026-02-25T14:21:24.707 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/95 2026-02-25T14:21:24.707 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/95 2026-02-25T14:21:24.707 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/95 2026-02-25T14:21:24.707 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/95 2026-02-25T14:21:24.707 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/95 2026-02-25T14:21:24.707 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/95 2026-02-25T14:21:24.707 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 81/95 2026-02-25T14:21:24.708 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 82/95 2026-02-25T14:21:24.708 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 83/95 2026-02-25T14:21:24.708 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 84/95 2026-02-25T14:21:24.708 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 85/95 2026-02-25T14:21:24.708 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 86/95 2026-02-25T14:21:24.708 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 87/95 2026-02-25T14:21:24.708 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 88/95 2026-02-25T14:21:24.708 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 89/95 2026-02-25T14:21:24.708 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 90/95 2026-02-25T14:21:24.708 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 91/95 2026-02-25T14:21:24.708 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 92/95 2026-02-25T14:21:24.708 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 93/95 2026-02-25T14:21:24.708 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 94/95 2026-02-25T14:21:24.768 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 95/95 2026-02-25T14:21:24.768 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:24.768 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:21:24.768 INFO:teuthology.orchestra.run.trial093.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-25T14:21:24.769 INFO:teuthology.orchestra.run.trial093.stdout: ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:24.769 INFO:teuthology.orchestra.run.trial093.stdout: ceph-grafana-dashboards-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:24.769 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:24.769 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:24.769 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:24.769 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:24.769 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:24.769 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:24.769 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-modules-core-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:24.769 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:24.769 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:24.769 INFO:teuthology.orchestra.run.trial093.stdout: ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:24.770 INFO:teuthology.orchestra.run.trial093.stdout: ceph-prometheus-alerts-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:24.770 INFO:teuthology.orchestra.run.trial093.stdout: ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:24.770 INFO:teuthology.orchestra.run.trial093.stdout: cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:21:24.770 INFO:teuthology.orchestra.run.trial093.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-25T14:21:24.770 INFO:teuthology.orchestra.run.trial093.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-25T14:21:24.770 INFO:teuthology.orchestra.run.trial093.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-25T14:21:24.770 INFO:teuthology.orchestra.run.trial093.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-25T14:21:24.770 INFO:teuthology.orchestra.run.trial093.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-25T14:21:24.770 INFO:teuthology.orchestra.run.trial093.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-25T14:21:24.770 INFO:teuthology.orchestra.run.trial093.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-25T14:21:24.770 INFO:teuthology.orchestra.run.trial093.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-25T14:21:24.771 INFO:teuthology.orchestra.run.trial093.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-25T14:21:24.771 INFO:teuthology.orchestra.run.trial093.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T14:21:24.771 INFO:teuthology.orchestra.run.trial093.stdout: lua-5.4.4-4.el9.x86_64 2026-02-25T14:21:24.771 INFO:teuthology.orchestra.run.trial093.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-25T14:21:24.771 INFO:teuthology.orchestra.run.trial093.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-25T14:21:24.771 INFO:teuthology.orchestra.run.trial093.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-25T14:21:24.771 INFO:teuthology.orchestra.run.trial093.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-25T14:21:24.771 INFO:teuthology.orchestra.run.trial093.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-25T14:21:24.771 INFO:teuthology.orchestra.run.trial093.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-25T14:21:24.771 INFO:teuthology.orchestra.run.trial093.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-25T14:21:24.771 INFO:teuthology.orchestra.run.trial093.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-25T14:21:24.771 INFO:teuthology.orchestra.run.trial093.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-25T14:21:24.771 INFO:teuthology.orchestra.run.trial093.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-25T14:21:24.772 INFO:teuthology.orchestra.run.trial093.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-25T14:21:24.772 INFO:teuthology.orchestra.run.trial093.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-25T14:21:24.772 INFO:teuthology.orchestra.run.trial093.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-25T14:21:24.772 INFO:teuthology.orchestra.run.trial093.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-25T14:21:24.772 INFO:teuthology.orchestra.run.trial093.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-25T14:21:24.772 INFO:teuthology.orchestra.run.trial093.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-25T14:21:24.772 INFO:teuthology.orchestra.run.trial093.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-25T14:21:24.772 INFO:teuthology.orchestra.run.trial093.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-25T14:21:24.772 INFO:teuthology.orchestra.run.trial093.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-25T14:21:24.772 INFO:teuthology.orchestra.run.trial093.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-25T14:21:24.772 INFO:teuthology.orchestra.run.trial093.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-25T14:21:24.772 INFO:teuthology.orchestra.run.trial093.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-25T14:21:24.773 INFO:teuthology.orchestra.run.trial093.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-25T14:21:24.773 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-25T14:21:24.773 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-25T14:21:24.773 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-25T14:21:24.773 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-25T14:21:24.773 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-25T14:21:24.773 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-25T14:21:24.773 INFO:teuthology.orchestra.run.trial093.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-25T14:21:24.773 INFO:teuthology.orchestra.run.trial093.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-25T14:21:24.773 INFO:teuthology.orchestra.run.trial093.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-25T14:21:24.773 INFO:teuthology.orchestra.run.trial093.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T14:21:24.773 INFO:teuthology.orchestra.run.trial093.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-25T14:21:24.774 INFO:teuthology.orchestra.run.trial093.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-25T14:21:24.774 INFO:teuthology.orchestra.run.trial093.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-25T14:21:24.774 INFO:teuthology.orchestra.run.trial093.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-25T14:21:24.774 INFO:teuthology.orchestra.run.trial093.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-25T14:21:24.774 INFO:teuthology.orchestra.run.trial093.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-25T14:21:24.774 INFO:teuthology.orchestra.run.trial093.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-25T14:21:24.774 INFO:teuthology.orchestra.run.trial093.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-25T14:21:24.774 INFO:teuthology.orchestra.run.trial093.stdout: python3-ply-3.11-14.el9.noarch 2026-02-25T14:21:24.774 INFO:teuthology.orchestra.run.trial093.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-25T14:21:24.774 INFO:teuthology.orchestra.run.trial093.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-25T14:21:24.774 INFO:teuthology.orchestra.run.trial093.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-25T14:21:24.774 INFO:teuthology.orchestra.run.trial093.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-25T14:21:24.774 INFO:teuthology.orchestra.run.trial093.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-25T14:21:24.775 INFO:teuthology.orchestra.run.trial093.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-25T14:21:24.775 INFO:teuthology.orchestra.run.trial093.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-25T14:21:24.775 INFO:teuthology.orchestra.run.trial093.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-25T14:21:24.775 INFO:teuthology.orchestra.run.trial093.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-25T14:21:24.775 INFO:teuthology.orchestra.run.trial093.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-25T14:21:24.775 INFO:teuthology.orchestra.run.trial093.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-25T14:21:24.775 INFO:teuthology.orchestra.run.trial093.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-25T14:21:24.775 INFO:teuthology.orchestra.run.trial093.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-25T14:21:24.775 INFO:teuthology.orchestra.run.trial093.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-25T14:21:24.775 INFO:teuthology.orchestra.run.trial093.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-25T14:21:24.775 INFO:teuthology.orchestra.run.trial093.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-25T14:21:24.775 INFO:teuthology.orchestra.run.trial093.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-25T14:21:24.776 INFO:teuthology.orchestra.run.trial093.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-25T14:21:24.776 INFO:teuthology.orchestra.run.trial093.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-25T14:21:24.776 INFO:teuthology.orchestra.run.trial093.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-25T14:21:24.776 INFO:teuthology.orchestra.run.trial093.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-25T14:21:24.776 INFO:teuthology.orchestra.run.trial093.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-25T14:21:24.776 INFO:teuthology.orchestra.run.trial093.stdout: unzip-6.0-59.el9.x86_64 2026-02-25T14:21:24.776 INFO:teuthology.orchestra.run.trial093.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-25T14:21:24.776 INFO:teuthology.orchestra.run.trial093.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-25T14:21:24.776 INFO:teuthology.orchestra.run.trial093.stdout: zip-3.0-35.el9.x86_64 2026-02-25T14:21:24.776 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:24.776 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:24.825 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-base 2026-02-25T14:21:25.146 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:51 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:25.211 INFO:teuthology.orchestra.run.trial093.stdout:Package ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:25.230 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:25.231 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:21:25.231 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:25.253 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install cephadm 2026-02-25T14:21:25.479 INFO:teuthology.orchestra.run.trial094.stdout:ceph-fuse-20.2.0-676.gc824a7fa.el9.x86_64.rpm 978 kB/s | 942 kB 00:00 2026-02-25T14:21:25.480 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:25.480 INFO:teuthology.orchestra.run.trial094.stdout:Total 977 kB/s | 942 kB 00:00 2026-02-25T14:21:25.480 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:21:25.485 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:21:25.485 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:21:25.518 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:21:25.518 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:21:25.574 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:51 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:25.608 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:21:25.626 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:25.639 INFO:teuthology.orchestra.run.trial093.stdout:Package cephadm-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:21:25.658 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:25.659 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:21:25.659 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:25.681 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T14:21:25.834 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:25.896 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:25.896 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:25.896 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:21:25.896 INFO:teuthology.orchestra.run.trial094.stdout: ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:25.896 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:25.897 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:25.924 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-volume 2026-02-25T14:21:26.006 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:52 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:26.091 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:26.091 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:26.092 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:21:26.092 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:26.092 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:21:26.092 INFO:teuthology.orchestra.run.trial093.stdout: ceph-immutable-object-cache x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 154 k 2026-02-25T14:21:26.092 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:26.092 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:21:26.092 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:26.092 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:21:26.092 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:26.092 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 154 k 2026-02-25T14:21:26.093 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 447 k 2026-02-25T14:21:26.093 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:21:26.246 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:52 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:26.311 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:21:26.329 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:26.330 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:21:26.330 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:26.351 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install librados-devel 2026-02-25T14:21:26.417 INFO:teuthology.orchestra.run.trial093.stdout:ceph-immutable-object-cache-20.2.0-676.gc824a7f 472 kB/s | 154 kB 00:00 2026-02-25T14:21:26.418 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:26.418 INFO:teuthology.orchestra.run.trial093.stdout:Total 471 kB/s | 154 kB 00:00 2026-02-25T14:21:26.418 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:21:26.423 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:21:26.423 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:21:26.454 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:21:26.454 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:21:26.510 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:21:26.512 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.e 1/1 2026-02-25T14:21:26.534 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.e 1/1 2026-02-25T14:21:26.534 INFO:teuthology.orchestra.run.trial093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:21:26.534 INFO:teuthology.orchestra.run.trial093.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-25T14:21:26.534 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:26.670 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:52 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:26.754 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:26.755 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:26.755 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:21:26.755 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:26.756 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:21:26.756 INFO:teuthology.orchestra.run.trial094.stdout: librados-devel x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 126 k 2026-02-25T14:21:26.756 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:26.756 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:21:26.756 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:26.756 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:21:26.756 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:26.756 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 126 k 2026-02-25T14:21:26.756 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 449 k 2026-02-25T14:21:26.756 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:21:26.831 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.e 1/1 2026-02-25T14:21:26.831 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:26.831 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:21:26.831 INFO:teuthology.orchestra.run.trial093.stdout: ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:26.831 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:26.831 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:26.858 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-mgr 2026-02-25T14:21:27.178 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:53 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:27.243 INFO:teuthology.orchestra.run.trial093.stdout:Package ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:27.262 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:27.262 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:21:27.262 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:27.284 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T14:21:27.340 INFO:teuthology.orchestra.run.trial094.stdout:librados-devel-20.2.0-676.gc824a7fa.el9.x86_64. 215 kB/s | 126 kB 00:00 2026-02-25T14:21:27.340 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:27.340 INFO:teuthology.orchestra.run.trial094.stdout:Total 215 kB/s | 126 kB 00:00 2026-02-25T14:21:27.340 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:21:27.343 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:21:27.343 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:21:27.375 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:21:27.375 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:21:27.431 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:21:27.448 INFO:teuthology.orchestra.run.trial094.stdout: Installing : librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:27.533 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:27.595 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:27.595 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:27.595 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:21:27.595 INFO:teuthology.orchestra.run.trial094.stdout: librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:27.595 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:27.595 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:27.598 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:53 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:27.624 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install libcephfs2 2026-02-25T14:21:27.666 INFO:teuthology.orchestra.run.trial093.stdout:Package ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:21:27.684 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:27.685 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:21:27.685 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:27.705 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T14:21:27.942 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:53 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:28.007 INFO:teuthology.orchestra.run.trial094.stdout:Package libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:28.023 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:54 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:28.026 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:28.026 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:21:28.026 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:28.048 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install libcephfs-devel 2026-02-25T14:21:28.089 INFO:teuthology.orchestra.run.trial093.stdout:Package ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:21:28.107 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:28.108 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:21:28.108 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:28.130 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-mgr-rook 2026-02-25T14:21:28.365 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:54 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:28.447 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:54 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:28.451 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:28.451 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:28.452 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:21:28.452 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:28.452 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:21:28.452 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs-devel x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 34 k 2026-02-25T14:21:28.452 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:28.452 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:21:28.452 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:28.452 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:21:28.452 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:28.452 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 34 k 2026-02-25T14:21:28.452 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 155 k 2026-02-25T14:21:28.452 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:21:28.512 INFO:teuthology.orchestra.run.trial093.stdout:Package ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:21:28.530 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:28.531 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:21:28.531 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:28.553 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T14:21:28.849 INFO:teuthology.orchestra.run.trial094.stdout:libcephfs-devel-20.2.0-676.gc824a7fa.el9.x86_64 87 kB/s | 34 kB 00:00 2026-02-25T14:21:28.849 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:28.849 INFO:teuthology.orchestra.run.trial094.stdout:Total 86 kB/s | 34 kB 00:00 2026-02-25T14:21:28.850 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:21:28.853 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:21:28.853 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:21:28.855 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:21:28.855 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:21:28.872 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:54 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:28.877 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:21:28.894 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:28.938 INFO:teuthology.orchestra.run.trial093.stdout:Package ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:21:28.956 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:28.957 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:28.957 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:21:28.958 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:28.978 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-fuse 2026-02-25T14:21:29.020 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:29.020 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:29.020 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:21:29.020 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:29.021 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:29.021 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:29.048 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install librados2 2026-02-25T14:21:29.298 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:55 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:29.360 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:55 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:29.384 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:29.384 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:29.384 INFO:teuthology.orchestra.run.trial093.stdout: Package Architecture Version Repository Size 2026-02-25T14:21:29.384 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:29.384 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:21:29.385 INFO:teuthology.orchestra.run.trial093.stdout: ceph-fuse x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 942 k 2026-02-25T14:21:29.385 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:29.385 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:21:29.385 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:29.385 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:21:29.385 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:29.385 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 942 k 2026-02-25T14:21:29.385 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 2.7 M 2026-02-25T14:21:29.385 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:21:29.425 INFO:teuthology.orchestra.run.trial094.stdout:Package librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:29.443 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:29.444 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:21:29.444 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:29.465 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install librbd1 2026-02-25T14:21:29.775 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:55 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:29.840 INFO:teuthology.orchestra.run.trial094.stdout:Package librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:29.858 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:29.859 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:21:29.859 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:29.879 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-rados 2026-02-25T14:21:30.036 INFO:teuthology.orchestra.run.trial093.stdout:ceph-fuse-20.2.0-676.gc824a7fa.el9.x86_64.rpm 1.4 MB/s | 942 kB 00:00 2026-02-25T14:21:30.037 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:30.037 INFO:teuthology.orchestra.run.trial093.stdout:Total 1.4 MB/s | 942 kB 00:00 2026-02-25T14:21:30.037 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:21:30.042 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:21:30.042 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:21:30.075 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:21:30.075 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:21:30.166 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:21:30.184 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:30.195 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:56 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:30.260 INFO:teuthology.orchestra.run.trial094.stdout:Package python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:30.279 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:30.279 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:21:30.279 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:30.300 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-rgw 2026-02-25T14:21:30.413 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:30.476 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:30.477 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:30.477 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:21:30.477 INFO:teuthology.orchestra.run.trial093.stdout: ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:30.477 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:30.477 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:30.504 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-volume 2026-02-25T14:21:30.619 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:56 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:30.684 INFO:teuthology.orchestra.run.trial094.stdout:Package python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:30.702 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:30.703 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:21:30.703 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:30.724 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-cephfs 2026-02-25T14:21:30.828 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:56 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:30.893 INFO:teuthology.orchestra.run.trial093.stdout:Package ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:21:30.912 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:30.912 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:21:30.913 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:30.934 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install librados-devel 2026-02-25T14:21:31.042 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:57 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:31.108 INFO:teuthology.orchestra.run.trial094.stdout:Package python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:31.127 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:31.127 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:21:31.127 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:31.149 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-rbd 2026-02-25T14:21:31.253 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:57 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:31.337 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:31.338 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:31.338 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:21:31.338 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:31.338 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:21:31.338 INFO:teuthology.orchestra.run.trial093.stdout: librados-devel x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 126 k 2026-02-25T14:21:31.338 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:31.338 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:21:31.338 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:31.338 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:21:31.339 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:31.339 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 126 k 2026-02-25T14:21:31.339 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 449 k 2026-02-25T14:21:31.339 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:21:31.467 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:57 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:31.534 INFO:teuthology.orchestra.run.trial094.stdout:Package python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:31.553 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:31.554 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:21:31.554 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:31.574 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install rbd-fuse 2026-02-25T14:21:31.729 INFO:teuthology.orchestra.run.trial090.stdout:(5/5): ceph-test-20.2.0-676.gc824a7fa.el9.x86_6 2.5 MB/s | 84 MB 00:33 2026-02-25T14:21:31.731 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:31.731 INFO:teuthology.orchestra.run.trial090.stdout:Total 2.5 MB/s | 85 MB 00:33 2026-02-25T14:21:31.756 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:21:31.767 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:21:31.767 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:21:31.800 INFO:teuthology.orchestra.run.trial093.stdout:librados-devel-20.2.0-676.gc824a7fa.el9.x86_64. 272 kB/s | 126 kB 00:00 2026-02-25T14:21:31.801 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:31.801 INFO:teuthology.orchestra.run.trial093.stdout:Total 271 kB/s | 126 kB 00:00 2026-02-25T14:21:31.802 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:21:31.804 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:21:31.804 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:21:31.836 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:21:31.837 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:21:31.891 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:57 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:31.892 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:21:31.909 INFO:teuthology.orchestra.run.trial093.stdout: Installing : librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:31.974 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:31.975 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:31.975 INFO:teuthology.orchestra.run.trial094.stdout: Package Architecture Version Repository Size 2026-02-25T14:21:31.975 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:31.975 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:21:31.975 INFO:teuthology.orchestra.run.trial094.stdout: rbd-fuse x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 91 k 2026-02-25T14:21:31.975 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:31.975 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:21:31.975 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:31.975 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:21:31.975 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:31.975 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 91 k 2026-02-25T14:21:31.975 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 238 k 2026-02-25T14:21:31.975 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:21:31.999 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:32.057 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:21:32.057 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:21:32.062 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:32.062 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:32.063 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:21:32.063 INFO:teuthology.orchestra.run.trial093.stdout: librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:32.063 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:32.063 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:32.091 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install libcephfs2 2026-02-25T14:21:32.371 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:21:32.376 INFO:teuthology.orchestra.run.trial090.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-25T14:21:32.381 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-25T14:21:32.389 INFO:teuthology.orchestra.run.trial090.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-25T14:21:32.390 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 4/5 2026-02-25T14:21:32.412 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:58 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:32.414 INFO:teuthology.orchestra.run.trial094.stdout:rbd-fuse-20.2.0-676.gc824a7fa.el9.x86_64.rpm 208 kB/s | 91 kB 00:00 2026-02-25T14:21:32.415 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:32.415 INFO:teuthology.orchestra.run.trial094.stdout:Total 207 kB/s | 91 kB 00:00 2026-02-25T14:21:32.415 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:21:32.420 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:21:32.420 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:21:32.450 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:21:32.450 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:21:32.477 INFO:teuthology.orchestra.run.trial093.stdout:Package libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:32.496 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:32.497 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:21:32.497 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:32.502 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:21:32.519 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install libcephfs-devel 2026-02-25T14:21:32.520 INFO:teuthology.orchestra.run.trial094.stdout: Installing : rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:32.604 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:32.667 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:32.667 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:32.667 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:21:32.667 INFO:teuthology.orchestra.run.trial094.stdout: rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:32.667 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:32.667 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:32.695 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install rbd-mirror 2026-02-25T14:21:32.834 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:58 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:32.919 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:32.919 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:32.919 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:21:32.919 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:32.919 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:21:32.919 INFO:teuthology.orchestra.run.trial093.stdout: libcephfs-devel x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 34 k 2026-02-25T14:21:32.919 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:32.919 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:21:32.920 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:32.920 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:21:32.920 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:32.920 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 34 k 2026-02-25T14:21:32.920 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 155 k 2026-02-25T14:21:32.920 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:21:33.027 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:59 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:33.109 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:33.110 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:33.110 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:21:33.110 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:33.110 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:21:33.110 INFO:teuthology.orchestra.run.trial094.stdout: rbd-mirror x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.9 M 2026-02-25T14:21:33.110 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:33.110 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:21:33.110 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:33.110 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:21:33.110 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:33.110 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 2.9 M 2026-02-25T14:21:33.110 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 11 M 2026-02-25T14:21:33.110 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:21:33.273 INFO:teuthology.orchestra.run.trial093.stdout:libcephfs-devel-20.2.0-676.gc824a7fa.el9.x86_64 97 kB/s | 34 kB 00:00 2026-02-25T14:21:33.273 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:33.274 INFO:teuthology.orchestra.run.trial093.stdout:Total 97 kB/s | 34 kB 00:00 2026-02-25T14:21:33.274 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:21:33.276 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:21:33.276 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:21:33.279 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:21:33.280 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:21:33.301 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:21:33.318 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:33.380 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:33.443 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:33.443 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:33.443 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:21:33.443 INFO:teuthology.orchestra.run.trial093.stdout: libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:33.443 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:33.443 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:33.473 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install librados2 2026-02-25T14:21:33.793 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:00:59 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:33.859 INFO:teuthology.orchestra.run.trial093.stdout:Package librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:33.878 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:33.878 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:21:33.879 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:33.901 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install librbd1 2026-02-25T14:21:33.947 INFO:teuthology.orchestra.run.trial094.stdout:rbd-mirror-20.2.0-676.gc824a7fa.el9.x86_64.rpm 3.5 MB/s | 2.9 MB 00:00 2026-02-25T14:21:33.947 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:33.947 INFO:teuthology.orchestra.run.trial094.stdout:Total 3.5 MB/s | 2.9 MB 00:00 2026-02-25T14:21:33.948 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:21:33.952 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:21:33.952 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:21:33.992 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:21:33.992 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:21:34.171 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:21:34.173 INFO:teuthology.orchestra.run.trial094.stdout: Installing : rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:34.195 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:34.195 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:21:34.196 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-25T14:21:34.196 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T14:21:34.196 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T14:21:34.196 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:34.223 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:01:00 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:34.288 INFO:teuthology.orchestra.run.trial093.stdout:Package librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:34.307 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:34.308 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:21:34.308 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:34.330 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install python3-rados 2026-02-25T14:21:34.500 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:34.501 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:34.501 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:21:34.501 INFO:teuthology.orchestra.run.trial094.stdout: rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:34.501 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:34.501 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:34.528 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install rbd-nbd 2026-02-25T14:21:34.650 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:01:00 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:34.715 INFO:teuthology.orchestra.run.trial093.stdout:Package python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:34.734 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:34.735 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:21:34.735 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:34.757 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install python3-rgw 2026-02-25T14:21:34.845 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:01:00 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:34.928 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:34.929 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:34.929 INFO:teuthology.orchestra.run.trial094.stdout: Package Architecture Version Repository Size 2026-02-25T14:21:34.929 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:34.929 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:21:34.929 INFO:teuthology.orchestra.run.trial094.stdout: rbd-nbd x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 180 k 2026-02-25T14:21:34.929 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:34.929 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:21:34.929 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:34.929 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:21:34.929 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:34.929 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 180 k 2026-02-25T14:21:34.929 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 498 k 2026-02-25T14:21:34.929 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:21:35.081 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:01:01 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:35.146 INFO:teuthology.orchestra.run.trial093.stdout:Package python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:35.165 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:35.165 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:21:35.166 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:35.188 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install python3-cephfs 2026-02-25T14:21:35.338 INFO:teuthology.orchestra.run.trial094.stdout:rbd-nbd-20.2.0-676.gc824a7fa.el9.x86_64.rpm 440 kB/s | 180 kB 00:00 2026-02-25T14:21:35.338 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:35.338 INFO:teuthology.orchestra.run.trial094.stdout:Total 439 kB/s | 180 kB 00:00 2026-02-25T14:21:35.338 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:21:35.343 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:21:35.343 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:21:35.374 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:21:35.374 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:21:35.431 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:21:35.448 INFO:teuthology.orchestra.run.trial094.stdout: Installing : rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:35.510 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:01:01 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:35.532 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:35.576 INFO:teuthology.orchestra.run.trial093.stdout:Package python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:35.594 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:35.595 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:35.595 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:35.595 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:21:35.595 INFO:teuthology.orchestra.run.trial094.stdout: rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:35.595 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:35.595 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:21:35.596 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:21:35.596 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:35.617 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install python3-rbd 2026-02-25T14:21:35.621 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-base-debuginfo 2026-02-25T14:21:35.931 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:01:01 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:35.942 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:01:01 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:35.997 INFO:teuthology.orchestra.run.trial093.stdout:Package python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:21:36.016 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:36.017 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:21:36.017 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:36.027 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:21:36.027 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:36.027 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:21:36.027 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:36.027 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:21:36.027 INFO:teuthology.orchestra.run.trial094.stdout: ceph-base-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 107 M 2026-02-25T14:21:36.027 INFO:teuthology.orchestra.run.trial094.stdout:Installing weak dependencies: 2026-02-25T14:21:36.028 INFO:teuthology.orchestra.run.trial094.stdout: ceph-debugsource x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 11 M 2026-02-25T14:21:36.028 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:36.028 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:21:36.028 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:21:36.028 INFO:teuthology.orchestra.run.trial094.stdout:Install 2 Packages 2026-02-25T14:21:36.028 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:21:36.028 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 118 M 2026-02-25T14:21:36.028 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 751 M 2026-02-25T14:21:36.028 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:21:36.038 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install rbd-fuse 2026-02-25T14:21:36.101 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 4/5 2026-02-25T14:21:36.120 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 5/5 2026-02-25T14:21:36.358 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:01:02 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:36.442 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 5/5 2026-02-25T14:21:36.442 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 1/5 2026-02-25T14:21:36.442 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 2/5 2026-02-25T14:21:36.442 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-25T14:21:36.442 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-25T14:21:36.443 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:36.443 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:36.443 INFO:teuthology.orchestra.run.trial093.stdout: Package Architecture Version Repository Size 2026-02-25T14:21:36.443 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:36.443 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:21:36.443 INFO:teuthology.orchestra.run.trial093.stdout: rbd-fuse x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 91 k 2026-02-25T14:21:36.443 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:36.443 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:21:36.443 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:36.443 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:21:36.443 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:36.444 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 91 k 2026-02-25T14:21:36.444 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 238 k 2026-02-25T14:21:36.444 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:21:36.501 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-25T14:21:36.501 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:21:36.501 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:21:36.501 INFO:teuthology.orchestra.run.trial090.stdout: ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:36.501 INFO:teuthology.orchestra.run.trial090.stdout: libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:36.501 INFO:teuthology.orchestra.run.trial090.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-25T14:21:36.501 INFO:teuthology.orchestra.run.trial090.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-25T14:21:36.501 INFO:teuthology.orchestra.run.trial090.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-25T14:21:36.501 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:21:36.502 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:21:36.538 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph 2026-02-25T14:21:36.854 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:02 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:36.860 INFO:teuthology.orchestra.run.trial093.stdout:rbd-fuse-20.2.0-676.gc824a7fa.el9.x86_64.rpm 219 kB/s | 91 kB 00:00 2026-02-25T14:21:36.861 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:36.861 INFO:teuthology.orchestra.run.trial093.stdout:Total 218 kB/s | 91 kB 00:00 2026-02-25T14:21:36.861 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:21:36.865 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:21:36.866 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:21:36.897 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:21:36.898 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:21:36.942 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:21:36.945 INFO:teuthology.orchestra.run.trial090.stdout:====================================================================================== 2026-02-25T14:21:36.945 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repository Size 2026-02-25T14:21:36.945 INFO:teuthology.orchestra.run.trial090.stdout:====================================================================================== 2026-02-25T14:21:36.945 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:21:36.945 INFO:teuthology.orchestra.run.trial090.stdout: ceph x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 6.5 k 2026-02-25T14:21:36.946 INFO:teuthology.orchestra.run.trial090.stdout:Installing dependencies: 2026-02-25T14:21:36.946 INFO:teuthology.orchestra.run.trial090.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-25T14:21:36.946 INFO:teuthology.orchestra.run.trial090.stdout: ceph-grafana-dashboards noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 43 k 2026-02-25T14:21:36.946 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mds x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.3 M 2026-02-25T14:21:36.946 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 961 k 2026-02-25T14:21:36.946 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr-modules-core noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 290 k 2026-02-25T14:21:36.946 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mon x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 5.0 M 2026-02-25T14:21:36.946 INFO:teuthology.orchestra.run.trial090.stdout: ceph-osd x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 17 M 2026-02-25T14:21:36.946 INFO:teuthology.orchestra.run.trial090.stdout: ceph-prometheus-alerts noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 17 k 2026-02-25T14:21:36.946 INFO:teuthology.orchestra.run.trial090.stdout: cephadm noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 1.0 M 2026-02-25T14:21:36.946 INFO:teuthology.orchestra.run.trial090.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-25T14:21:36.946 INFO:teuthology.orchestra.run.trial090.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-25T14:21:36.947 INFO:teuthology.orchestra.run.trial090.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-25T14:21:36.947 INFO:teuthology.orchestra.run.trial090.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-25T14:21:36.947 INFO:teuthology.orchestra.run.trial090.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-25T14:21:36.947 INFO:teuthology.orchestra.run.trial090.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-25T14:21:36.947 INFO:teuthology.orchestra.run.trial090.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-25T14:21:36.947 INFO:teuthology.orchestra.run.trial090.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-25T14:21:36.947 INFO:teuthology.orchestra.run.trial090.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-25T14:21:36.947 INFO:teuthology.orchestra.run.trial090.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-25T14:21:36.947 INFO:teuthology.orchestra.run.trial090.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-25T14:21:36.947 INFO:teuthology.orchestra.run.trial090.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-25T14:21:36.948 INFO:teuthology.orchestra.run.trial090.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-25T14:21:36.948 INFO:teuthology.orchestra.run.trial090.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-25T14:21:36.948 INFO:teuthology.orchestra.run.trial090.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-25T14:21:36.948 INFO:teuthology.orchestra.run.trial090.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-25T14:21:36.948 INFO:teuthology.orchestra.run.trial090.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-25T14:21:36.948 INFO:teuthology.orchestra.run.trial090.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-25T14:21:36.948 INFO:teuthology.orchestra.run.trial090.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-25T14:21:36.948 INFO:teuthology.orchestra.run.trial090.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-25T14:21:36.948 INFO:teuthology.orchestra.run.trial090.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-25T14:21:36.948 INFO:teuthology.orchestra.run.trial090.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-25T14:21:36.948 INFO:teuthology.orchestra.run.trial090.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-25T14:21:36.949 INFO:teuthology.orchestra.run.trial090.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-25T14:21:36.949 INFO:teuthology.orchestra.run.trial090.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-25T14:21:36.949 INFO:teuthology.orchestra.run.trial090.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-25T14:21:36.949 INFO:teuthology.orchestra.run.trial090.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-25T14:21:36.949 INFO:teuthology.orchestra.run.trial090.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-25T14:21:36.949 INFO:teuthology.orchestra.run.trial090.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-25T14:21:36.949 INFO:teuthology.orchestra.run.trial090.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-25T14:21:36.949 INFO:teuthology.orchestra.run.trial090.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-25T14:21:36.949 INFO:teuthology.orchestra.run.trial090.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-25T14:21:36.949 INFO:teuthology.orchestra.run.trial090.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-25T14:21:36.950 INFO:teuthology.orchestra.run.trial090.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-25T14:21:36.950 INFO:teuthology.orchestra.run.trial090.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-25T14:21:36.950 INFO:teuthology.orchestra.run.trial090.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-25T14:21:36.950 INFO:teuthology.orchestra.run.trial090.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-25T14:21:36.950 INFO:teuthology.orchestra.run.trial090.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-25T14:21:36.950 INFO:teuthology.orchestra.run.trial090.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-25T14:21:36.950 INFO:teuthology.orchestra.run.trial090.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-25T14:21:36.950 INFO:teuthology.orchestra.run.trial090.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-25T14:21:36.950 INFO:teuthology.orchestra.run.trial090.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-25T14:21:36.950 INFO:teuthology.orchestra.run.trial090.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-25T14:21:36.950 INFO:teuthology.orchestra.run.trial090.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-25T14:21:36.950 INFO:teuthology.orchestra.run.trial090.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-25T14:21:36.950 INFO:teuthology.orchestra.run.trial090.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-25T14:21:36.951 INFO:teuthology.orchestra.run.trial090.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-25T14:21:36.951 INFO:teuthology.orchestra.run.trial090.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-25T14:21:36.951 INFO:teuthology.orchestra.run.trial090.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-25T14:21:36.951 INFO:teuthology.orchestra.run.trial090.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-25T14:21:36.951 INFO:teuthology.orchestra.run.trial090.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-25T14:21:36.951 INFO:teuthology.orchestra.run.trial090.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-25T14:21:36.951 INFO:teuthology.orchestra.run.trial090.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-25T14:21:36.951 INFO:teuthology.orchestra.run.trial090.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-25T14:21:36.951 INFO:teuthology.orchestra.run.trial090.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-25T14:21:36.951 INFO:teuthology.orchestra.run.trial090.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-25T14:21:36.951 INFO:teuthology.orchestra.run.trial090.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-25T14:21:36.951 INFO:teuthology.orchestra.run.trial090.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-25T14:21:36.951 INFO:teuthology.orchestra.run.trial090.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-25T14:21:36.952 INFO:teuthology.orchestra.run.trial090.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-25T14:21:36.952 INFO:teuthology.orchestra.run.trial090.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-25T14:21:36.952 INFO:teuthology.orchestra.run.trial090.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-25T14:21:36.952 INFO:teuthology.orchestra.run.trial090.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-25T14:21:36.952 INFO:teuthology.orchestra.run.trial090.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-25T14:21:36.952 INFO:teuthology.orchestra.run.trial090.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-25T14:21:36.952 INFO:teuthology.orchestra.run.trial090.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-25T14:21:36.952 INFO:teuthology.orchestra.run.trial090.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-25T14:21:36.952 INFO:teuthology.orchestra.run.trial090.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-25T14:21:36.952 INFO:teuthology.orchestra.run.trial090.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-25T14:21:36.952 INFO:teuthology.orchestra.run.trial090.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-25T14:21:36.952 INFO:teuthology.orchestra.run.trial090.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-25T14:21:36.952 INFO:teuthology.orchestra.run.trial090.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-25T14:21:36.953 INFO:teuthology.orchestra.run.trial090.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-25T14:21:36.953 INFO:teuthology.orchestra.run.trial090.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-25T14:21:36.953 INFO:teuthology.orchestra.run.trial090.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-25T14:21:36.953 INFO:teuthology.orchestra.run.trial090.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-25T14:21:36.953 INFO:teuthology.orchestra.run.trial090.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-25T14:21:36.953 INFO:teuthology.orchestra.run.trial090.stdout:Installing weak dependencies: 2026-02-25T14:21:36.953 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr-cephadm noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 173 k 2026-02-25T14:21:36.953 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr-dashboard noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 11 M 2026-02-25T14:21:36.953 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr-diskprediction-local noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 7.4 M 2026-02-25T14:21:36.953 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr-k8sevents noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 22 k 2026-02-25T14:21:36.953 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr-rook noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 50 k 2026-02-25T14:21:36.954 INFO:teuthology.orchestra.run.trial090.stdout: ceph-volume noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 297 k 2026-02-25T14:21:36.954 INFO:teuthology.orchestra.run.trial090.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-25T14:21:36.954 INFO:teuthology.orchestra.run.trial090.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-25T14:21:36.954 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:21:36.954 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:21:36.954 INFO:teuthology.orchestra.run.trial090.stdout:====================================================================================== 2026-02-25T14:21:36.954 INFO:teuthology.orchestra.run.trial090.stdout:Install 95 Packages 2026-02-25T14:21:36.954 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:21:36.955 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:21:36.956 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 101 M 2026-02-25T14:21:36.956 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 561 M 2026-02-25T14:21:36.956 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:21:36.966 INFO:teuthology.orchestra.run.trial093.stdout: Installing : rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:37.057 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:37.119 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:37.119 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:37.119 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:21:37.119 INFO:teuthology.orchestra.run.trial093.stdout: rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:37.119 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:37.120 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:37.149 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install rbd-mirror 2026-02-25T14:21:37.261 INFO:teuthology.orchestra.run.trial189.stdout:(5/5): ceph-test-20.2.0-676.gc824a7fa.el9.x86_6 2.1 MB/s | 84 MB 00:39 2026-02-25T14:21:37.262 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:37.262 INFO:teuthology.orchestra.run.trial189.stdout:Total 2.1 MB/s | 85 MB 00:39 2026-02-25T14:21:37.286 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:21:37.296 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:21:37.296 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:21:37.467 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:01:03 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:37.551 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:37.552 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:37.552 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:21:37.552 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:37.552 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:21:37.552 INFO:teuthology.orchestra.run.trial093.stdout: rbd-mirror x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.9 M 2026-02-25T14:21:37.552 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:37.552 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:21:37.552 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:37.552 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:21:37.552 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:37.553 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 2.9 M 2026-02-25T14:21:37.553 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 11 M 2026-02-25T14:21:37.553 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:21:37.583 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:21:37.583 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:21:37.704 INFO:teuthology.orchestra.run.trial090.stdout:(1/95): ceph-20.2.0-676.gc824a7fa.el9.x86_64.rp 23 kB/s | 6.5 kB 00:00 2026-02-25T14:21:37.895 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:21:37.901 INFO:teuthology.orchestra.run.trial189.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-25T14:21:37.905 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-25T14:21:37.913 INFO:teuthology.orchestra.run.trial189.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-25T14:21:37.914 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 4/5 2026-02-25T14:21:38.508 INFO:teuthology.orchestra.run.trial093.stdout:rbd-mirror-20.2.0-676.gc824a7fa.el9.x86_64.rpm 3.0 MB/s | 2.9 MB 00:00 2026-02-25T14:21:38.509 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:38.510 INFO:teuthology.orchestra.run.trial093.stdout:Total 3.0 MB/s | 2.9 MB 00:00 2026-02-25T14:21:38.510 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:21:38.514 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:21:38.514 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:21:38.553 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:21:38.553 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:21:38.624 INFO:teuthology.orchestra.run.trial090.stdout:(2/95): ceph-mgr-20.2.0-676.gc824a7fa.el9.x86_6 798 kB/s | 961 kB 00:01 2026-02-25T14:21:38.679 INFO:teuthology.orchestra.run.trial094.stdout:(1/2): ceph-debugsource-20.2.0-676.gc824a7fa.el 4.2 MB/s | 11 MB 00:02 2026-02-25T14:21:38.733 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:21:38.734 INFO:teuthology.orchestra.run.trial093.stdout: Installing : rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:38.757 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:38.757 INFO:teuthology.orchestra.run.trial093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:21:38.757 INFO:teuthology.orchestra.run.trial093.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-25T14:21:38.757 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T14:21:38.757 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T14:21:38.757 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:39.059 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:39.059 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:39.059 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:21:39.059 INFO:teuthology.orchestra.run.trial093.stdout: rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:39.059 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:39.060 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:39.086 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install rbd-nbd 2026-02-25T14:21:39.361 INFO:teuthology.orchestra.run.trial090.stdout:(3/95): ceph-mds-20.2.0-676.gc824a7fa.el9.x86_6 1.2 MB/s | 2.3 MB 00:01 2026-02-25T14:21:39.411 INFO:teuthology.orchestra.run.trial090.stdout:(4/95): ceph-grafana-dashboards-20.2.0-676.gc82 869 kB/s | 43 kB 00:00 2026-02-25T14:21:39.412 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:01:05 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:39.496 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:39.497 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:39.497 INFO:teuthology.orchestra.run.trial093.stdout: Package Architecture Version Repository Size 2026-02-25T14:21:39.497 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:39.497 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:21:39.497 INFO:teuthology.orchestra.run.trial093.stdout: rbd-nbd x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 180 k 2026-02-25T14:21:39.497 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:39.497 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:21:39.497 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:39.497 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:21:39.498 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:39.498 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 180 k 2026-02-25T14:21:39.498 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 498 k 2026-02-25T14:21:39.498 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:21:39.512 INFO:teuthology.orchestra.run.trial090.stdout:(5/95): ceph-mgr-cephadm-20.2.0-676.gc824a7fa.e 1.7 MB/s | 173 kB 00:00 2026-02-25T14:21:39.889 INFO:teuthology.orchestra.run.trial093.stdout:rbd-nbd-20.2.0-676.gc824a7fa.el9.x86_64.rpm 458 kB/s | 180 kB 00:00 2026-02-25T14:21:39.889 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:39.890 INFO:teuthology.orchestra.run.trial093.stdout:Total 457 kB/s | 180 kB 00:00 2026-02-25T14:21:39.890 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:21:39.895 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:21:39.895 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:21:39.926 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:21:39.926 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:21:39.983 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:21:40.001 INFO:teuthology.orchestra.run.trial093.stdout: Installing : rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:40.089 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:40.152 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:21:40.152 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:40.153 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:21:40.153 INFO:teuthology.orchestra.run.trial093.stdout: rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:40.153 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:40.153 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:21:40.181 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-base-debuginfo 2026-02-25T14:21:40.500 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:01:06 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:40.585 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:21:40.585 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:40.585 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:21:40.585 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:40.585 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:21:40.585 INFO:teuthology.orchestra.run.trial093.stdout: ceph-base-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 107 M 2026-02-25T14:21:40.586 INFO:teuthology.orchestra.run.trial093.stdout:Installing weak dependencies: 2026-02-25T14:21:40.586 INFO:teuthology.orchestra.run.trial093.stdout: ceph-debugsource x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 11 M 2026-02-25T14:21:40.586 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:40.586 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:21:40.586 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:21:40.586 INFO:teuthology.orchestra.run.trial093.stdout:Install 2 Packages 2026-02-25T14:21:40.586 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:21:40.586 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 118 M 2026-02-25T14:21:40.586 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 751 M 2026-02-25T14:21:40.586 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:21:41.623 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 4/5 2026-02-25T14:21:41.642 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 5/5 2026-02-25T14:21:41.976 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 5/5 2026-02-25T14:21:41.976 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 1/5 2026-02-25T14:21:41.976 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 2/5 2026-02-25T14:21:41.976 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-25T14:21:41.976 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-25T14:21:42.035 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-25T14:21:42.035 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:21:42.035 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:21:42.035 INFO:teuthology.orchestra.run.trial189.stdout: ceph-test-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:42.036 INFO:teuthology.orchestra.run.trial189.stdout: libcephsqlite-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:21:42.036 INFO:teuthology.orchestra.run.trial189.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-25T14:21:42.036 INFO:teuthology.orchestra.run.trial189.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-25T14:21:42.036 INFO:teuthology.orchestra.run.trial189.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-25T14:21:42.036 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:21:42.036 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:21:42.070 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph 2026-02-25T14:21:42.387 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:08 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:21:42.475 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout:====================================================================================== 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repository Size 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout:====================================================================================== 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: ceph x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 6.5 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout:Installing dependencies: 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: ceph-grafana-dashboards noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 43 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mds x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.3 M 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 961 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-modules-core noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 290 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mon x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 5.0 M 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: ceph-osd x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 17 M 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: ceph-prometheus-alerts noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 17 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: cephadm noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 1.0 M 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-25T14:21:42.478 INFO:teuthology.orchestra.run.trial189.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-25T14:21:42.479 INFO:teuthology.orchestra.run.trial189.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout:Installing weak dependencies: 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-cephadm noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 173 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-dashboard noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 11 M 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-diskprediction-local noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 7.4 M 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-k8sevents noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 22 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-rook noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 50 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: ceph-volume noarch 2:20.2.0-676.gc824a7fa.el9 ceph-noarch 297 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout:====================================================================================== 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout:Install 95 Packages 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 101 M 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 561 M 2026-02-25T14:21:42.480 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:21:43.318 INFO:teuthology.orchestra.run.trial189.stdout:(1/95): ceph-20.2.0-676.gc824a7fa.el9.x86_64.rp 23 kB/s | 6.5 kB 00:00 2026-02-25T14:21:43.666 INFO:teuthology.orchestra.run.trial093.stdout:(1/2): ceph-debugsource-20.2.0-676.gc824a7fa.el 3.6 MB/s | 11 MB 00:03 2026-02-25T14:21:43.963 INFO:teuthology.orchestra.run.trial090.stdout:(6/95): ceph-mon-20.2.0-676.gc824a7fa.el9.x86_6 826 kB/s | 5.0 MB 00:06 2026-02-25T14:21:44.047 INFO:teuthology.orchestra.run.trial189.stdout:(2/95): ceph-mon-20.2.0-676.gc824a7fa.el9.x86_6 6.9 MB/s | 5.0 MB 00:00 2026-02-25T14:21:44.263 INFO:teuthology.orchestra.run.trial189.stdout:(3/95): ceph-mgr-20.2.0-676.gc824a7fa.el9.x86_6 785 kB/s | 961 kB 00:01 2026-02-25T14:21:44.323 INFO:teuthology.orchestra.run.trial189.stdout:(4/95): ceph-grafana-dashboards-20.2.0-676.gc82 729 kB/s | 43 kB 00:00 2026-02-25T14:21:44.487 INFO:teuthology.orchestra.run.trial189.stdout:(5/95): ceph-mgr-cephadm-20.2.0-676.gc824a7fa.e 1.0 MB/s | 173 kB 00:00 2026-02-25T14:21:44.635 INFO:teuthology.orchestra.run.trial189.stdout:(6/95): ceph-mds-20.2.0-676.gc824a7fa.el9.x86_6 1.5 MB/s | 2.3 MB 00:01 2026-02-25T14:21:45.095 INFO:teuthology.orchestra.run.trial189.stdout:(7/95): ceph-osd-20.2.0-676.gc824a7fa.el9.x86_6 16 MB/s | 17 MB 00:01 2026-02-25T14:21:45.152 INFO:teuthology.orchestra.run.trial189.stdout:(8/95): ceph-mgr-k8sevents-20.2.0-676.gc824a7fa 391 kB/s | 22 kB 00:00 2026-02-25T14:21:45.211 INFO:teuthology.orchestra.run.trial189.stdout:(9/95): ceph-mgr-modules-core-20.2.0-676.gc824a 4.7 MB/s | 290 kB 00:00 2026-02-25T14:21:45.268 INFO:teuthology.orchestra.run.trial189.stdout:(10/95): ceph-mgr-rook-20.2.0-676.gc824a7fa.el9 892 kB/s | 50 kB 00:00 2026-02-25T14:21:45.280 INFO:teuthology.orchestra.run.trial090.stdout:(7/95): ceph-mgr-dashboard-20.2.0-676.gc824a7fa 1.8 MB/s | 11 MB 00:05 2026-02-25T14:21:45.322 INFO:teuthology.orchestra.run.trial189.stdout:(11/95): ceph-prometheus-alerts-20.2.0-676.gc82 321 kB/s | 17 kB 00:00 2026-02-25T14:21:45.335 INFO:teuthology.orchestra.run.trial090.stdout:(8/95): ceph-mgr-k8sevents-20.2.0-676.gc824a7fa 403 kB/s | 22 kB 00:00 2026-02-25T14:21:45.384 INFO:teuthology.orchestra.run.trial189.stdout:(12/95): ceph-volume-20.2.0-676.gc824a7fa.el9.n 4.6 MB/s | 297 kB 00:00 2026-02-25T14:21:45.588 INFO:teuthology.orchestra.run.trial090.stdout:(9/95): ceph-mgr-modules-core-20.2.0-676.gc824a 1.1 MB/s | 290 kB 00:00 2026-02-25T14:21:45.689 INFO:teuthology.orchestra.run.trial090.stdout:(10/95): ceph-mgr-rook-20.2.0-676.gc824a7fa.el9 496 kB/s | 50 kB 00:00 2026-02-25T14:21:45.741 INFO:teuthology.orchestra.run.trial090.stdout:(11/95): ceph-prometheus-alerts-20.2.0-676.gc82 332 kB/s | 17 kB 00:00 2026-02-25T14:21:45.791 INFO:teuthology.orchestra.run.trial189.stdout:(13/95): cephadm-20.2.0-676.gc824a7fa.el9.noarc 2.5 MB/s | 1.0 MB 00:00 2026-02-25T14:21:46.164 INFO:teuthology.orchestra.run.trial090.stdout:(12/95): ceph-volume-20.2.0-676.gc824a7fa.el9.n 704 kB/s | 297 kB 00:00 2026-02-25T14:21:46.748 INFO:teuthology.orchestra.run.trial189.stdout:(14/95): cryptsetup-2.8.1-3.el9.x86_64.rpm 367 kB/s | 351 kB 00:00 2026-02-25T14:21:46.907 INFO:teuthology.orchestra.run.trial189.stdout:(15/95): ledmon-libs-1.1.0-3.el9.x86_64.rpm 256 kB/s | 40 kB 00:00 2026-02-25T14:21:47.081 INFO:teuthology.orchestra.run.trial090.stdout:(13/95): cephadm-20.2.0-676.gc824a7fa.el9.noarc 1.1 MB/s | 1.0 MB 00:00 2026-02-25T14:21:47.143 INFO:teuthology.orchestra.run.trial189.stdout:(16/95): libconfig-1.7.2-9.el9.x86_64.rpm 305 kB/s | 72 kB 00:00 2026-02-25T14:21:47.220 INFO:teuthology.orchestra.run.trial090.stdout:(14/95): cryptsetup-2.8.1-3.el9.x86_64.rpm 2.5 MB/s | 351 kB 00:00 2026-02-25T14:21:47.240 INFO:teuthology.orchestra.run.trial090.stdout:(15/95): ledmon-libs-1.1.0-3.el9.x86_64.rpm 2.0 MB/s | 40 kB 00:00 2026-02-25T14:21:47.262 INFO:teuthology.orchestra.run.trial090.stdout:(16/95): libconfig-1.7.2-9.el9.x86_64.rpm 3.3 MB/s | 72 kB 00:00 2026-02-25T14:21:47.312 INFO:teuthology.orchestra.run.trial090.stdout:(17/95): libgfortran-11.5.0-14.el9.x86_64.rpm 15 MB/s | 794 kB 00:00 2026-02-25T14:21:47.336 INFO:teuthology.orchestra.run.trial090.stdout:(18/95): libquadmath-11.5.0-14.el9.x86_64.rpm 7.9 MB/s | 184 kB 00:00 2026-02-25T14:21:47.358 INFO:teuthology.orchestra.run.trial090.stdout:(19/95): python3-cffi-1.14.5-5.el9.x86_64.rpm 11 MB/s | 253 kB 00:00 2026-02-25T14:21:47.404 INFO:teuthology.orchestra.run.trial090.stdout:(20/95): python3-cryptography-36.0.1-5.el9.x86_ 28 MB/s | 1.2 MB 00:00 2026-02-25T14:21:47.425 INFO:teuthology.orchestra.run.trial090.stdout:(21/95): python3-ply-3.11-14.el9.noarch.rpm 4.9 MB/s | 106 kB 00:00 2026-02-25T14:21:47.447 INFO:teuthology.orchestra.run.trial090.stdout:(22/95): python3-pycparser-2.20-6.el9.noarch.rp 6.1 MB/s | 135 kB 00:00 2026-02-25T14:21:47.469 INFO:teuthology.orchestra.run.trial090.stdout:(23/95): python3-pyparsing-2.4.7-9.el9.noarch.r 6.7 MB/s | 150 kB 00:00 2026-02-25T14:21:47.491 INFO:teuthology.orchestra.run.trial090.stdout:(24/95): python3-requests-2.25.1-10.el9.noarch. 5.6 MB/s | 126 kB 00:00 2026-02-25T14:21:47.514 INFO:teuthology.orchestra.run.trial090.stdout:(25/95): python3-urllib3-1.26.5-7.el9.noarch.rp 9.6 MB/s | 218 kB 00:00 2026-02-25T14:21:47.536 INFO:teuthology.orchestra.run.trial090.stdout:(26/95): unzip-6.0-59.el9.x86_64.rpm 7.9 MB/s | 182 kB 00:00 2026-02-25T14:21:47.558 INFO:teuthology.orchestra.run.trial090.stdout:(27/95): zip-3.0-35.el9.x86_64.rpm 12 MB/s | 266 kB 00:00 2026-02-25T14:21:47.968 INFO:teuthology.orchestra.run.trial090.stdout:(28/95): flexiblas-3.0.4-9.el9.x86_64.rpm 72 kB/s | 30 kB 00:00 2026-02-25T14:21:48.162 INFO:teuthology.orchestra.run.trial189.stdout:(17/95): libgfortran-11.5.0-14.el9.x86_64.rpm 779 kB/s | 794 kB 00:01 2026-02-25T14:21:48.477 INFO:teuthology.orchestra.run.trial189.stdout:(18/95): libquadmath-11.5.0-14.el9.x86_64.rpm 587 kB/s | 184 kB 00:00 2026-02-25T14:21:48.832 INFO:teuthology.orchestra.run.trial090.stdout:(29/95): ceph-mgr-diskprediction-local-20.2.0-6 1.5 MB/s | 7.4 MB 00:04 2026-02-25T14:21:48.833 INFO:teuthology.orchestra.run.trial189.stdout:(19/95): ceph-mgr-diskprediction-local-20.2.0-6 1.8 MB/s | 7.4 MB 00:04 2026-02-25T14:21:48.870 INFO:teuthology.orchestra.run.trial189.stdout:(20/95): python3-cffi-1.14.5-5.el9.x86_64.rpm 643 kB/s | 253 kB 00:00 2026-02-25T14:21:48.984 INFO:teuthology.orchestra.run.trial090.stdout:(30/95): ceph-osd-20.2.0-676.gc824a7fa.el9.x86_ 1.6 MB/s | 17 MB 00:10 2026-02-25T14:21:49.233 INFO:teuthology.orchestra.run.trial090.stdout:(31/95): flexiblas-openblas-openmp-3.0.4-9.el9. 37 kB/s | 15 kB 00:00 2026-02-25T14:21:49.551 INFO:teuthology.orchestra.run.trial090.stdout:(32/95): lua-5.4.4-4.el9.x86_64.rpm 594 kB/s | 188 kB 00:00 2026-02-25T14:21:49.580 INFO:teuthology.orchestra.run.trial189.stdout:(21/95): python3-ply-3.11-14.el9.noarch.rpm 150 kB/s | 106 kB 00:00 2026-02-25T14:21:49.633 INFO:teuthology.orchestra.run.trial090.stdout:(33/95): openblas-0.3.29-1.el9.x86_64.rpm 508 kB/s | 42 kB 00:00 2026-02-25T14:21:49.701 INFO:teuthology.orchestra.run.trial090.stdout:(34/95): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 343 kB/s | 246 kB 00:00 2026-02-25T14:21:49.894 INFO:teuthology.orchestra.run.trial189.stdout:(22/95): python3-pycparser-2.20-6.el9.noarch.rp 431 kB/s | 135 kB 00:00 2026-02-25T14:21:50.208 INFO:teuthology.orchestra.run.trial189.stdout:(23/95): python3-pyparsing-2.4.7-9.el9.noarch.r 480 kB/s | 150 kB 00:00 2026-02-25T14:21:50.405 INFO:teuthology.orchestra.run.trial189.stdout:(24/95): ceph-mgr-dashboard-20.2.0-676.gc824a7f 1.8 MB/s | 11 MB 00:05 2026-02-25T14:21:50.406 INFO:teuthology.orchestra.run.trial189.stdout:(25/95): python3-cryptography-36.0.1-5.el9.x86_ 811 kB/s | 1.2 MB 00:01 2026-02-25T14:21:50.443 INFO:teuthology.orchestra.run.trial189.stdout:(26/95): python3-requests-2.25.1-10.el9.noarch. 536 kB/s | 126 kB 00:00 2026-02-25T14:21:50.493 INFO:teuthology.orchestra.run.trial090.stdout:(35/95): protobuf-3.14.0-17.el9.x86_64.rpm 1.3 MB/s | 1.0 MB 00:00 2026-02-25T14:21:50.719 INFO:teuthology.orchestra.run.trial189.stdout:(27/95): unzip-6.0-59.el9.x86_64.rpm 581 kB/s | 182 kB 00:00 2026-02-25T14:21:50.799 INFO:teuthology.orchestra.run.trial189.stdout:(28/95): python3-urllib3-1.26.5-7.el9.noarch.rp 553 kB/s | 218 kB 00:00 2026-02-25T14:21:50.805 INFO:teuthology.orchestra.run.trial189.stdout:(29/95): flexiblas-3.0.4-9.el9.x86_64.rpm 346 kB/s | 30 kB 00:00 2026-02-25T14:21:50.887 INFO:teuthology.orchestra.run.trial189.stdout:(30/95): flexiblas-openblas-openmp-3.0.4-9.el9. 181 kB/s | 15 kB 00:00 2026-02-25T14:21:50.905 INFO:teuthology.orchestra.run.trial090.stdout:(36/95): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2026-02-25T14:21:50.991 INFO:teuthology.orchestra.run.trial090.stdout:(37/95): python3-devel-3.9.25-3.el9.x86_64.rpm 2.8 MB/s | 244 kB 00:00 2026-02-25T14:21:51.076 INFO:teuthology.orchestra.run.trial090.stdout:(38/95): python3-jinja2-2.11.3-8.el9.noarch.rpm 2.9 MB/s | 249 kB 00:00 2026-02-25T14:21:51.160 INFO:teuthology.orchestra.run.trial090.stdout:(39/95): python3-jmespath-1.0.1-1.el9.noarch.rp 563 kB/s | 48 kB 00:00 2026-02-25T14:21:51.203 INFO:teuthology.orchestra.run.trial189.stdout:(31/95): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 779 kB/s | 246 kB 00:00 2026-02-25T14:21:51.245 INFO:teuthology.orchestra.run.trial090.stdout:(40/95): python3-libstoragemgmt-1.10.1-1.el9.x8 2.0 MB/s | 177 kB 00:00 2026-02-25T14:21:51.320 INFO:teuthology.orchestra.run.trial189.stdout:(32/95): zip-3.0-35.el9.x86_64.rpm 303 kB/s | 266 kB 00:00 2026-02-25T14:21:51.335 INFO:teuthology.orchestra.run.trial090.stdout:(41/95): python3-lxml-4.6.5-3.el9.x86_64.rpm 14 MB/s | 1.2 MB 00:00 2026-02-25T14:21:51.404 INFO:teuthology.orchestra.run.trial090.stdout:(42/95): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 891 kB/s | 3.0 MB 00:03 2026-02-25T14:21:51.405 INFO:teuthology.orchestra.run.trial189.stdout:(33/95): openblas-0.3.29-1.el9.x86_64.rpm 505 kB/s | 42 kB 00:00 2026-02-25T14:21:51.419 INFO:teuthology.orchestra.run.trial090.stdout:(43/95): python3-markupsafe-1.1.1-12.el9.x86_64 414 kB/s | 35 kB 00:00 2026-02-25T14:21:51.443 INFO:teuthology.orchestra.run.trial189.stdout:(34/95): lua-5.4.4-4.el9.x86_64.rpm 784 kB/s | 188 kB 00:00 2026-02-25T14:21:51.504 INFO:teuthology.orchestra.run.trial090.stdout:(44/95): python3-numpy-f2py-1.23.5-2.el9.x86_64 5.0 MB/s | 442 kB 00:00 2026-02-25T14:21:51.589 INFO:teuthology.orchestra.run.trial090.stdout:(45/95): python3-packaging-20.9-5.el9.noarch.rp 917 kB/s | 77 kB 00:00 2026-02-25T14:21:51.674 INFO:teuthology.orchestra.run.trial090.stdout:(46/95): python3-protobuf-3.14.0-17.el9.noarch. 3.1 MB/s | 267 kB 00:00 2026-02-25T14:21:51.758 INFO:teuthology.orchestra.run.trial090.stdout:(47/95): python3-pyasn1-0.4.8-7.el9.noarch.rpm 1.8 MB/s | 157 kB 00:00 2026-02-25T14:21:51.844 INFO:teuthology.orchestra.run.trial090.stdout:(48/95): python3-pyasn1-modules-0.4.8-7.el9.noa 3.2 MB/s | 277 kB 00:00 2026-02-25T14:21:51.929 INFO:teuthology.orchestra.run.trial090.stdout:(49/95): python3-requests-oauthlib-1.3.0-12.el9 631 kB/s | 54 kB 00:00 2026-02-25T14:21:52.616 INFO:teuthology.orchestra.run.trial189.stdout:(35/95): protobuf-3.14.0-17.el9.x86_64.rpm 877 kB/s | 1.0 MB 00:01 2026-02-25T14:21:53.159 INFO:teuthology.orchestra.run.trial090.stdout:(50/95): python3-scipy-1.9.3-2.el9.x86_64.rpm 16 MB/s | 19 MB 00:01 2026-02-25T14:21:53.243 INFO:teuthology.orchestra.run.trial090.stdout:(51/95): python3-toml-0.10.2-6.el9.noarch.rpm 493 kB/s | 42 kB 00:00 2026-02-25T14:21:53.328 INFO:teuthology.orchestra.run.trial090.stdout:(52/95): xmlsec1-1.2.29-13.el9.x86_64.rpm 2.2 MB/s | 189 kB 00:00 2026-02-25T14:21:53.412 INFO:teuthology.orchestra.run.trial090.stdout:(53/95): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 1.0 MB/s | 90 kB 00:00 2026-02-25T14:21:53.445 INFO:teuthology.orchestra.run.trial090.stdout:(54/95): lua-devel-5.4.4-4.el9.x86_64.rpm 681 kB/s | 22 kB 00:00 2026-02-25T14:21:53.503 INFO:teuthology.orchestra.run.trial090.stdout:(55/95): protobuf-compiler-3.14.0-17.el9.x86_64 15 MB/s | 862 kB 00:00 2026-02-25T14:21:54.038 INFO:teuthology.orchestra.run.trial090.stdout:(56/95): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.0 MB/s | 551 kB 00:00 2026-02-25T14:21:54.067 INFO:teuthology.orchestra.run.trial090.stdout:(57/95): python3-numpy-1.23.5-2.el9.x86_64.rpm 2.3 MB/s | 6.1 MB 00:02 2026-02-25T14:21:54.166 INFO:teuthology.orchestra.run.trial090.stdout:(58/95): grpc-data-1.46.7-10.el9.noarch.rpm 151 kB/s | 19 kB 00:00 2026-02-25T14:21:54.169 INFO:teuthology.orchestra.run.trial090.stdout:(59/95): luarocks-3.9.2-5.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2026-02-25T14:21:54.252 INFO:teuthology.orchestra.run.trial189.stdout:(36/95): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 886 kB/s | 3.0 MB 00:03 2026-02-25T14:21:54.268 INFO:teuthology.orchestra.run.trial090.stdout:(60/95): python3-autocommand-2.2.2-8.el9.noarch 299 kB/s | 29 kB 00:00 2026-02-25T14:21:54.295 INFO:teuthology.orchestra.run.trial090.stdout:(61/95): python3-backports-tarfile-1.2.0-1.el9. 2.2 MB/s | 60 kB 00:00 2026-02-25T14:21:54.313 INFO:teuthology.orchestra.run.trial090.stdout:(62/95): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 2.4 MB/s | 43 kB 00:00 2026-02-25T14:21:54.361 INFO:teuthology.orchestra.run.trial090.stdout:(63/95): python3-cachetools-4.2.4-1.el9.noarch. 679 kB/s | 32 kB 00:00 2026-02-25T14:21:54.412 INFO:teuthology.orchestra.run.trial090.stdout:(64/95): python3-certifi-2023.05.07-4.el9.noarc 275 kB/s | 14 kB 00:00 2026-02-25T14:21:54.482 INFO:teuthology.orchestra.run.trial090.stdout:(65/95): python3-cheroot-10.0.1-4.el9.noarch.rp 2.4 MB/s | 173 kB 00:00 2026-02-25T14:21:54.570 INFO:teuthology.orchestra.run.trial189.stdout:(37/95): python3-devel-3.9.25-3.el9.x86_64.rpm 767 kB/s | 244 kB 00:00 2026-02-25T14:21:54.682 INFO:teuthology.orchestra.run.trial090.stdout:(66/95): python3-asyncssh-2.13.2-5.el9.noarch.r 1.0 MB/s | 548 kB 00:00 2026-02-25T14:21:54.711 INFO:teuthology.orchestra.run.trial090.stdout:(67/95): python3-google-auth-2.45.0-1.el9.noarc 8.5 MB/s | 254 kB 00:00 2026-02-25T14:21:54.726 INFO:teuthology.orchestra.run.trial090.stdout:(68/95): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2026-02-25T14:21:54.821 INFO:teuthology.orchestra.run.trial090.stdout:(69/95): python3-grpcio-tools-1.46.7-10.el9.x86 1.5 MB/s | 144 kB 00:00 2026-02-25T14:21:54.889 INFO:teuthology.orchestra.run.trial189.stdout:(38/95): python3-jinja2-2.11.3-8.el9.noarch.rpm 781 kB/s | 249 kB 00:00 2026-02-25T14:21:54.907 INFO:teuthology.orchestra.run.trial090.stdout:(70/95): python3-influxdb-5.3.1-1.el9.noarch.rp 1.6 MB/s | 139 kB 00:00 2026-02-25T14:21:54.956 INFO:teuthology.orchestra.run.trial090.stdout:(71/95): python3-isodate-0.6.1-3.el9.noarch.rpm 1.1 MB/s | 56 kB 00:00 2026-02-25T14:21:54.970 INFO:teuthology.orchestra.run.trial189.stdout:(39/95): openblas-openmp-0.3.29-1.el9.x86_64.rp 1.5 MB/s | 5.3 MB 00:03 2026-02-25T14:21:54.972 INFO:teuthology.orchestra.run.trial189.stdout:(40/95): python3-jmespath-1.0.1-1.el9.noarch.rp 572 kB/s | 48 kB 00:00 2026-02-25T14:21:54.978 INFO:teuthology.orchestra.run.trial090.stdout:(72/95): python3-jaraco-8.2.1-3.el9.noarch.rpm 469 kB/s | 11 kB 00:00 2026-02-25T14:21:55.066 INFO:teuthology.orchestra.run.trial189.stdout:(41/95): python3-lxml-4.6.5-3.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2026-02-25T14:21:55.091 INFO:teuthology.orchestra.run.trial090.stdout:(73/95): python3-jaraco-classes-3.2.1-5.el9.noa 157 kB/s | 18 kB 00:00 2026-02-25T14:21:55.112 INFO:teuthology.orchestra.run.trial090.stdout:(74/95): python3-jaraco-collections-3.0.0-8.el9 1.1 MB/s | 23 kB 00:00 2026-02-25T14:21:55.143 INFO:teuthology.orchestra.run.trial090.stdout:(75/95): python3-grpcio-1.46.7-10.el9.x86_64.rp 4.7 MB/s | 2.0 MB 00:00 2026-02-25T14:21:55.150 INFO:teuthology.orchestra.run.trial189.stdout:(42/95): python3-markupsafe-1.1.1-12.el9.x86_64 412 kB/s | 35 kB 00:00 2026-02-25T14:21:55.211 INFO:teuthology.orchestra.run.trial189.stdout:(43/95): python3-libstoragemgmt-1.10.1-1.el9.x8 736 kB/s | 177 kB 00:00 2026-02-25T14:21:55.212 INFO:teuthology.orchestra.run.trial090.stdout:(76/95): python3-jaraco-functools-3.5.0-2.el9.n 278 kB/s | 19 kB 00:00 2026-02-25T14:21:55.228 INFO:teuthology.orchestra.run.trial090.stdout:(77/95): python3-jaraco-context-6.0.1-3.el9.noa 169 kB/s | 20 kB 00:00 2026-02-25T14:21:55.235 INFO:teuthology.orchestra.run.trial090.stdout:(78/95): python3-jaraco-text-4.0.0-2.el9.noarch 1.1 MB/s | 26 kB 00:00 2026-02-25T14:21:55.308 INFO:teuthology.orchestra.run.trial090.stdout:(79/95): python3-more-itertools-8.12.0-2.el9.no 1.1 MB/s | 79 kB 00:00 2026-02-25T14:21:55.369 INFO:teuthology.orchestra.run.trial090.stdout:(80/95): python3-msgpack-1.0.3-2.el9.x86_64.rpm 1.4 MB/s | 86 kB 00:00 2026-02-25T14:21:55.391 INFO:teuthology.orchestra.run.trial090.stdout:(81/95): python3-kubernetes-26.1.0-3.el9.noarch 6.3 MB/s | 1.0 MB 00:00 2026-02-25T14:21:55.420 INFO:teuthology.orchestra.run.trial090.stdout:(82/95): python3-portend-3.1.0-2.el9.noarch.rpm 560 kB/s | 16 kB 00:00 2026-02-25T14:21:55.478 INFO:teuthology.orchestra.run.trial090.stdout:(83/95): python3-natsort-7.1.1-5.el9.noarch.rpm 531 kB/s | 58 kB 00:00 2026-02-25T14:21:55.483 INFO:teuthology.orchestra.run.trial090.stdout:(84/95): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.4 MB/s | 90 kB 00:00 2026-02-25T14:21:55.536 INFO:teuthology.orchestra.run.trial090.stdout:(85/95): python3-repoze-lru-0.7-16.el9.noarch.r 529 kB/s | 31 kB 00:00 2026-02-25T14:21:55.566 INFO:teuthology.orchestra.run.trial189.stdout:(44/95): python3-numpy-1.23.5-2.el9.x86_64.rpm 15 MB/s | 6.1 MB 00:00 2026-02-25T14:21:55.574 INFO:teuthology.orchestra.run.trial090.stdout:(86/95): python3-rsa-4.9-2.el9.noarch.rpm 1.5 MB/s | 59 kB 00:00 2026-02-25T14:21:55.607 INFO:teuthology.orchestra.run.trial090.stdout:(87/95): python3-routes-2.5.1-5.el9.noarch.rpm 1.5 MB/s | 188 kB 00:00 2026-02-25T14:21:55.646 INFO:teuthology.orchestra.run.trial090.stdout:(88/95): openblas-openmp-0.3.29-1.el9.x86_64.rp 900 kB/s | 5.3 MB 00:06 2026-02-25T14:21:55.651 INFO:teuthology.orchestra.run.trial189.stdout:(45/95): python3-packaging-20.9-5.el9.noarch.rp 915 kB/s | 77 kB 00:00 2026-02-25T14:21:55.652 INFO:teuthology.orchestra.run.trial090.stdout:(89/95): python3-saml-1.16.0-1.el9.noarch.rpm 1.6 MB/s | 125 kB 00:00 2026-02-25T14:21:55.654 INFO:teuthology.orchestra.run.trial090.stdout:(90/95): python3-tempora-5.0.0-2.el9.noarch.rpm 759 kB/s | 36 kB 00:00 2026-02-25T14:21:55.670 INFO:teuthology.orchestra.run.trial090.stdout:(91/95): python3-websocket-client-1.2.3-2.el9.n 4.8 MB/s | 90 kB 00:00 2026-02-25T14:21:55.686 INFO:teuthology.orchestra.run.trial090.stdout:(92/95): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 1.5 MB/s | 48 kB 00:00 2026-02-25T14:21:55.710 INFO:teuthology.orchestra.run.trial090.stdout:(93/95): python3-zc-lockfile-2.0-10.el9.noarch. 824 kB/s | 20 kB 00:00 2026-02-25T14:21:55.725 INFO:teuthology.orchestra.run.trial090.stdout:(94/95): python3-typing-extensions-4.15.0-1.el9 1.1 MB/s | 86 kB 00:00 2026-02-25T14:21:55.736 INFO:teuthology.orchestra.run.trial189.stdout:(46/95): python3-protobuf-3.14.0-17.el9.noarch. 3.1 MB/s | 267 kB 00:00 2026-02-25T14:21:55.748 INFO:teuthology.orchestra.run.trial090.stdout:(95/95): python3-xmltodict-0.12.0-15.el9.noarch 287 kB/s | 22 kB 00:00 2026-02-25T14:21:55.753 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:55.753 INFO:teuthology.orchestra.run.trial090.stdout:Total 5.4 MB/s | 101 MB 00:18 2026-02-25T14:21:55.765 INFO:teuthology.orchestra.run.trial189.stdout:(47/95): python3-numpy-f2py-1.23.5-2.el9.x86_64 798 kB/s | 442 kB 00:00 2026-02-25T14:21:55.820 INFO:teuthology.orchestra.run.trial189.stdout:(48/95): python3-pyasn1-0.4.8-7.el9.noarch.rpm 1.8 MB/s | 157 kB 00:00 2026-02-25T14:21:55.905 INFO:teuthology.orchestra.run.trial189.stdout:(49/95): python3-requests-oauthlib-1.3.0-12.el9 637 kB/s | 54 kB 00:00 2026-02-25T14:21:56.083 INFO:teuthology.orchestra.run.trial189.stdout:(50/95): python3-pyasn1-modules-0.4.8-7.el9.noa 871 kB/s | 277 kB 00:00 2026-02-25T14:21:56.167 INFO:teuthology.orchestra.run.trial189.stdout:(51/95): python3-toml-0.10.2-6.el9.noarch.rpm 500 kB/s | 42 kB 00:00 2026-02-25T14:21:56.175 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:21:56.212 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:21:56.212 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:21:56.406 INFO:teuthology.orchestra.run.trial189.stdout:(52/95): xmlsec1-1.2.29-13.el9.x86_64.rpm 788 kB/s | 189 kB 00:00 2026-02-25T14:21:56.567 INFO:teuthology.orchestra.run.trial189.stdout:(53/95): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 561 kB/s | 90 kB 00:00 2026-02-25T14:21:56.646 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:21:56.646 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:21:56.655 INFO:teuthology.orchestra.run.trial189.stdout:(54/95): lua-devel-5.4.4-4.el9.x86_64.rpm 254 kB/s | 22 kB 00:00 2026-02-25T14:21:56.801 INFO:teuthology.orchestra.run.trial189.stdout:(55/95): protobuf-compiler-3.14.0-17.el9.x86_64 5.8 MB/s | 862 kB 00:00 2026-02-25T14:21:57.002 INFO:teuthology.orchestra.run.trial189.stdout:(56/95): abseil-cpp-20211102.0-4.el9.x86_64.rpm 2.7 MB/s | 551 kB 00:00 2026-02-25T14:21:57.062 INFO:teuthology.orchestra.run.trial189.stdout:(57/95): grpc-data-1.46.7-10.el9.noarch.rpm 324 kB/s | 19 kB 00:00 2026-02-25T14:21:57.085 INFO:teuthology.orchestra.run.trial189.stdout:(58/95): luarocks-3.9.2-5.el9.noarch.rpm 6.5 MB/s | 151 kB 00:00 2026-02-25T14:21:57.115 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:21:57.117 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/95 2026-02-25T14:21:57.121 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/95 2026-02-25T14:21:57.130 INFO:teuthology.orchestra.run.trial189.stdout:(59/95): python3-asyncssh-2.13.2-5.el9.noarch.r 12 MB/s | 548 kB 00:00 2026-02-25T14:21:57.142 INFO:teuthology.orchestra.run.trial090.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/95 2026-02-25T14:21:57.177 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 4/95 2026-02-25T14:21:57.187 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/95 2026-02-25T14:21:57.192 INFO:teuthology.orchestra.run.trial090.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 6/95 2026-02-25T14:21:57.195 INFO:teuthology.orchestra.run.trial090.stdout: Installing : lua-5.4.4-4.el9.x86_64 7/95 2026-02-25T14:21:57.199 INFO:teuthology.orchestra.run.trial090.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 8/95 2026-02-25T14:21:57.219 INFO:teuthology.orchestra.run.trial189.stdout:(60/95): python3-scipy-1.9.3-2.el9.x86_64.rpm 15 MB/s | 19 MB 00:01 2026-02-25T14:21:57.222 INFO:teuthology.orchestra.run.trial090.stdout: Installing : unzip-6.0-59.el9.x86_64 9/95 2026-02-25T14:21:57.235 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 10/95 2026-02-25T14:21:57.238 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 11/95 2026-02-25T14:21:57.244 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 12/95 2026-02-25T14:21:57.246 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 13/95 2026-02-25T14:21:57.255 INFO:teuthology.orchestra.run.trial189.stdout:(61/95): python3-autocommand-2.2.2-8.el9.noarch 236 kB/s | 29 kB 00:00 2026-02-25T14:21:57.255 INFO:teuthology.orchestra.run.trial189.stdout:(62/95): python3-backports-tarfile-1.2.0-1.el9. 1.6 MB/s | 60 kB 00:00 2026-02-25T14:21:57.257 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 14/95 2026-02-25T14:21:57.263 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 15/95 2026-02-25T14:21:57.288 INFO:teuthology.orchestra.run.trial090.stdout: Installing : zip-3.0-35.el9.x86_64 16/95 2026-02-25T14:21:57.291 INFO:teuthology.orchestra.run.trial090.stdout: Installing : luarocks-3.9.2-5.el9.noarch 17/95 2026-02-25T14:21:57.298 INFO:teuthology.orchestra.run.trial090.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 18/95 2026-02-25T14:21:57.349 INFO:teuthology.orchestra.run.trial090.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 19/95 2026-02-25T14:21:57.362 INFO:teuthology.orchestra.run.trial189.stdout:(63/95): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 408 kB/s | 43 kB 00:00 2026-02-25T14:21:57.362 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 20/95 2026-02-25T14:21:57.363 INFO:teuthology.orchestra.run.trial189.stdout:(64/95): python3-cachetools-4.2.4-1.el9.noarch. 302 kB/s | 32 kB 00:00 2026-02-25T14:21:57.367 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-rsa-4.9-2.el9.noarch 21/95 2026-02-25T14:21:57.371 INFO:teuthology.orchestra.run.trial090.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 22/95 2026-02-25T14:21:57.376 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 23/95 2026-02-25T14:21:57.381 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 24/95 2026-02-25T14:21:57.384 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 25/95 2026-02-25T14:21:57.399 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 26/95 2026-02-25T14:21:57.403 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 27/95 2026-02-25T14:21:57.408 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 28/95 2026-02-25T14:21:57.419 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 29/95 2026-02-25T14:21:57.428 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 30/95 2026-02-25T14:21:57.435 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 31/95 2026-02-25T14:21:57.455 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 32/95 2026-02-25T14:21:57.466 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 33/95 2026-02-25T14:21:57.481 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 34/95 2026-02-25T14:21:57.486 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 35/95 2026-02-25T14:21:57.494 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 36/95 2026-02-25T14:21:57.534 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 37/95 2026-02-25T14:21:57.672 INFO:teuthology.orchestra.run.trial189.stdout:(65/95): python3-certifi-2023.05.07-4.el9.noarc 46 kB/s | 14 kB 00:00 2026-02-25T14:21:57.673 INFO:teuthology.orchestra.run.trial189.stdout:(66/95): python3-cheroot-10.0.1-4.el9.noarch.rp 559 kB/s | 173 kB 00:00 2026-02-25T14:21:57.788 INFO:teuthology.orchestra.run.trial189.stdout:(67/95): python3-cherrypy-18.6.1-2.el9.noarch.r 3.0 MB/s | 358 kB 00:00 2026-02-25T14:21:57.808 INFO:teuthology.orchestra.run.trial189.stdout:(68/95): python3-google-auth-2.45.0-1.el9.noarc 1.8 MB/s | 254 kB 00:00 2026-02-25T14:21:57.839 INFO:teuthology.orchestra.run.trial189.stdout:(69/95): python3-grpcio-1.46.7-10.el9.x86_64.rp 40 MB/s | 2.0 MB 00:00 2026-02-25T14:21:57.839 INFO:teuthology.orchestra.run.trial189.stdout:(70/95): python3-grpcio-tools-1.46.7-10.el9.x86 4.5 MB/s | 144 kB 00:00 2026-02-25T14:21:57.860 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 38/95 2026-02-25T14:21:57.862 INFO:teuthology.orchestra.run.trial189.stdout:(71/95): python3-influxdb-5.3.1-1.el9.noarch.rp 6.0 MB/s | 139 kB 00:00 2026-02-25T14:21:57.862 INFO:teuthology.orchestra.run.trial189.stdout:(72/95): python3-isodate-0.6.1-3.el9.noarch.rpm 2.4 MB/s | 56 kB 00:00 2026-02-25T14:21:57.870 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 39/95 2026-02-25T14:21:57.874 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 40/95 2026-02-25T14:21:57.880 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 41/95 2026-02-25T14:21:57.882 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 42/95 2026-02-25T14:21:57.884 INFO:teuthology.orchestra.run.trial189.stdout:(73/95): python3-jaraco-8.2.1-3.el9.noarch.rpm 485 kB/s | 11 kB 00:00 2026-02-25T14:21:57.884 INFO:teuthology.orchestra.run.trial189.stdout:(74/95): python3-jaraco-classes-3.2.1-5.el9.noa 796 kB/s | 18 kB 00:00 2026-02-25T14:21:57.908 INFO:teuthology.orchestra.run.trial189.stdout:(75/95): python3-jaraco-collections-3.0.0-8.el9 969 kB/s | 23 kB 00:00 2026-02-25T14:21:57.908 INFO:teuthology.orchestra.run.trial189.stdout:(76/95): python3-jaraco-context-6.0.1-3.el9.noa 827 kB/s | 20 kB 00:00 2026-02-25T14:21:57.909 INFO:teuthology.orchestra.run.trial090.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 43/95 2026-02-25T14:21:57.931 INFO:teuthology.orchestra.run.trial189.stdout:(77/95): python3-jaraco-functools-3.5.0-2.el9.n 875 kB/s | 19 kB 00:00 2026-02-25T14:21:57.931 INFO:teuthology.orchestra.run.trial189.stdout:(78/95): python3-jaraco-text-4.0.0-2.el9.noarch 1.2 MB/s | 26 kB 00:00 2026-02-25T14:21:57.953 INFO:teuthology.orchestra.run.trial090.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 44/95 2026-02-25T14:21:57.959 INFO:teuthology.orchestra.run.trial189.stdout:(79/95): python3-kubernetes-26.1.0-3.el9.noarch 36 MB/s | 1.0 MB 00:00 2026-02-25T14:21:57.959 INFO:teuthology.orchestra.run.trial189.stdout:(80/95): python3-more-itertools-8.12.0-2.el9.no 2.7 MB/s | 79 kB 00:00 2026-02-25T14:21:57.960 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 45/95 2026-02-25T14:21:57.963 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 46/95 2026-02-25T14:21:57.969 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 47/95 2026-02-25T14:21:57.973 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 48/95 2026-02-25T14:21:57.980 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 49/95 2026-02-25T14:21:57.981 INFO:teuthology.orchestra.run.trial189.stdout:(81/95): python3-msgpack-1.0.3-2.el9.x86_64.rpm 3.8 MB/s | 86 kB 00:00 2026-02-25T14:21:57.982 INFO:teuthology.orchestra.run.trial189.stdout:(82/95): python3-natsort-7.1.1-5.el9.noarch.rpm 2.5 MB/s | 58 kB 00:00 2026-02-25T14:21:57.983 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 50/95 2026-02-25T14:21:58.004 INFO:teuthology.orchestra.run.trial189.stdout:(83/95): python3-portend-3.1.0-2.el9.noarch.rpm 739 kB/s | 16 kB 00:00 2026-02-25T14:21:58.004 INFO:teuthology.orchestra.run.trial189.stdout:(84/95): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.9 MB/s | 90 kB 00:00 2026-02-25T14:21:58.011 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 51/95 2026-02-25T14:21:58.021 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 52/95 2026-02-25T14:21:58.026 INFO:teuthology.orchestra.run.trial189.stdout:(85/95): python3-repoze-lru-0.7-16.el9.noarch.r 1.4 MB/s | 31 kB 00:00 2026-02-25T14:21:58.027 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 53/95 2026-02-25T14:21:58.028 INFO:teuthology.orchestra.run.trial189.stdout:(86/95): python3-routes-2.5.1-5.el9.noarch.rpm 7.9 MB/s | 188 kB 00:00 2026-02-25T14:21:58.034 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 54/95 2026-02-25T14:21:58.068 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 55/95 2026-02-25T14:21:58.244 INFO:teuthology.orchestra.run.trial189.stdout:(87/95): python3-rsa-4.9-2.el9.noarch.rpm 271 kB/s | 59 kB 00:00 2026-02-25T14:21:58.245 INFO:teuthology.orchestra.run.trial189.stdout:(88/95): python3-saml-1.16.0-1.el9.noarch.rpm 576 kB/s | 125 kB 00:00 2026-02-25T14:21:58.294 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 56/95 2026-02-25T14:21:58.318 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 57/95 2026-02-25T14:21:58.321 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 58/95 2026-02-25T14:21:58.380 INFO:teuthology.orchestra.run.trial090.stdout: Installing : openblas-0.3.29-1.el9.x86_64 59/95 2026-02-25T14:21:58.382 INFO:teuthology.orchestra.run.trial090.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 60/95 2026-02-25T14:21:58.404 INFO:teuthology.orchestra.run.trial090.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 61/95 2026-02-25T14:21:58.462 INFO:teuthology.orchestra.run.trial189.stdout:(89/95): python3-tempora-5.0.0-2.el9.noarch.rpm 164 kB/s | 36 kB 00:00 2026-02-25T14:21:58.463 INFO:teuthology.orchestra.run.trial189.stdout:(90/95): python3-typing-extensions-4.15.0-1.el9 397 kB/s | 86 kB 00:00 2026-02-25T14:21:58.515 INFO:teuthology.orchestra.run.trial189.stdout:(91/95): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2026-02-25T14:21:58.516 INFO:teuthology.orchestra.run.trial189.stdout:(92/95): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 919 kB/s | 48 kB 00:00 2026-02-25T14:21:58.537 INFO:teuthology.orchestra.run.trial189.stdout:(93/95): python3-xmltodict-0.12.0-15.el9.noarch 1.0 MB/s | 22 kB 00:00 2026-02-25T14:21:58.539 INFO:teuthology.orchestra.run.trial189.stdout:(94/95): python3-zc-lockfile-2.0-10.el9.noarch. 846 kB/s | 20 kB 00:00 2026-02-25T14:21:58.731 INFO:teuthology.orchestra.run.trial090.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 62/95 2026-02-25T14:21:58.805 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 63/95 2026-02-25T14:21:59.394 INFO:teuthology.orchestra.run.trial189.stdout:(95/95): python3-babel-2.9.1-2.el9.noarch.rpm 901 kB/s | 6.0 MB 00:06 2026-02-25T14:21:59.398 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:21:59.398 INFO:teuthology.orchestra.run.trial189.stdout:Total 6.0 MB/s | 101 MB 00:16 2026-02-25T14:21:59.481 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 64/95 2026-02-25T14:21:59.499 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 65/95 2026-02-25T14:21:59.509 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 66/95 2026-02-25T14:21:59.515 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/95 2026-02-25T14:21:59.529 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-ply-3.11-14.el9.noarch 68/95 2026-02-25T14:21:59.544 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 69/95 2026-02-25T14:21:59.620 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 70/95 2026-02-25T14:21:59.629 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 71/95 2026-02-25T14:21:59.653 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 72/95 2026-02-25T14:21:59.683 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 73/95 2026-02-25T14:21:59.733 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 74/95 2026-02-25T14:21:59.739 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 75/95 2026-02-25T14:21:59.743 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/95 2026-02-25T14:21:59.744 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 77/95 2026-02-25T14:21:59.761 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 78/95 2026-02-25T14:21:59.762 INFO:teuthology.orchestra.run.trial090.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-25T14:21:59.762 INFO:teuthology.orchestra.run.trial090.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-25T14:21:59.762 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:21:59.769 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 78/95 2026-02-25T14:21:59.795 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 78/95 2026-02-25T14:21:59.796 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-25T14:21:59.796 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:21:59.814 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:21:59.831 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 79/95 2026-02-25T14:21:59.852 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:21:59.852 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:21:59.895 INFO:teuthology.orchestra.run.trial090.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 80/95 2026-02-25T14:21:59.897 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 81/95 2026-02-25T14:21:59.911 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 81/95 2026-02-25T14:21:59.911 INFO:teuthology.orchestra.run.trial090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:21:59.911 INFO:teuthology.orchestra.run.trial090.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-25T14:21:59.911 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:00.288 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:22:00.288 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:22:00.583 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 82/95 2026-02-25T14:22:00.608 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 82/95 2026-02-25T14:22:00.608 INFO:teuthology.orchestra.run.trial090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:22:00.608 INFO:teuthology.orchestra.run.trial090.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-25T14:22:00.608 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T14:22:00.609 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T14:22:00.609 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:00.667 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 83/95 2026-02-25T14:22:00.669 INFO:teuthology.orchestra.run.trial090.stdout: Installing : cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 83/95 2026-02-25T14:22:00.675 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-prometheus-alerts-2:20.2.0-676.gc824a7fa.el9. 84/95 2026-02-25T14:22:00.699 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-grafana-dashboards-2:20.2.0-676.gc824a7fa.el9 85/95 2026-02-25T14:22:00.700 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 86/95 2026-02-25T14:22:00.761 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:22:00.764 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/95 2026-02-25T14:22:00.768 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/95 2026-02-25T14:22:00.790 INFO:teuthology.orchestra.run.trial189.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/95 2026-02-25T14:22:00.825 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 4/95 2026-02-25T14:22:00.834 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/95 2026-02-25T14:22:00.840 INFO:teuthology.orchestra.run.trial189.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 6/95 2026-02-25T14:22:00.843 INFO:teuthology.orchestra.run.trial189.stdout: Installing : lua-5.4.4-4.el9.x86_64 7/95 2026-02-25T14:22:00.848 INFO:teuthology.orchestra.run.trial189.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 8/95 2026-02-25T14:22:00.871 INFO:teuthology.orchestra.run.trial189.stdout: Installing : unzip-6.0-59.el9.x86_64 9/95 2026-02-25T14:22:00.884 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 10/95 2026-02-25T14:22:00.888 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 11/95 2026-02-25T14:22:00.895 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 12/95 2026-02-25T14:22:00.897 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 13/95 2026-02-25T14:22:00.908 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 14/95 2026-02-25T14:22:00.914 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 15/95 2026-02-25T14:22:00.939 INFO:teuthology.orchestra.run.trial189.stdout: Installing : zip-3.0-35.el9.x86_64 16/95 2026-02-25T14:22:00.942 INFO:teuthology.orchestra.run.trial189.stdout: Installing : luarocks-3.9.2-5.el9.noarch 17/95 2026-02-25T14:22:00.950 INFO:teuthology.orchestra.run.trial189.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 18/95 2026-02-25T14:22:01.002 INFO:teuthology.orchestra.run.trial189.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 19/95 2026-02-25T14:22:01.015 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 20/95 2026-02-25T14:22:01.017 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-rsa-4.9-2.el9.noarch 21/95 2026-02-25T14:22:01.022 INFO:teuthology.orchestra.run.trial189.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 22/95 2026-02-25T14:22:01.027 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 23/95 2026-02-25T14:22:01.032 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 24/95 2026-02-25T14:22:01.035 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 25/95 2026-02-25T14:22:01.049 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 26/95 2026-02-25T14:22:01.054 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 27/95 2026-02-25T14:22:01.059 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 28/95 2026-02-25T14:22:01.070 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 29/95 2026-02-25T14:22:01.079 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 30/95 2026-02-25T14:22:01.086 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 31/95 2026-02-25T14:22:01.105 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 32/95 2026-02-25T14:22:01.116 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 33/95 2026-02-25T14:22:01.132 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 34/95 2026-02-25T14:22:01.136 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 35/95 2026-02-25T14:22:01.144 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 36/95 2026-02-25T14:22:01.184 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 37/95 2026-02-25T14:22:01.509 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 38/95 2026-02-25T14:22:01.519 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 39/95 2026-02-25T14:22:01.523 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 40/95 2026-02-25T14:22:01.529 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 41/95 2026-02-25T14:22:01.531 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 42/95 2026-02-25T14:22:01.558 INFO:teuthology.orchestra.run.trial189.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 43/95 2026-02-25T14:22:01.601 INFO:teuthology.orchestra.run.trial189.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 44/95 2026-02-25T14:22:01.609 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 45/95 2026-02-25T14:22:01.612 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 46/95 2026-02-25T14:22:01.618 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 47/95 2026-02-25T14:22:01.621 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 48/95 2026-02-25T14:22:01.628 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 49/95 2026-02-25T14:22:01.632 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 50/95 2026-02-25T14:22:01.659 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 51/95 2026-02-25T14:22:01.669 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 52/95 2026-02-25T14:22:01.675 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 53/95 2026-02-25T14:22:01.681 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 54/95 2026-02-25T14:22:01.716 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 55/95 2026-02-25T14:22:01.848 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 86/95 2026-02-25T14:22:01.853 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noar 87/95 2026-02-25T14:22:01.940 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 56/95 2026-02-25T14:22:01.964 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 57/95 2026-02-25T14:22:01.968 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 58/95 2026-02-25T14:22:02.026 INFO:teuthology.orchestra.run.trial189.stdout: Installing : openblas-0.3.29-1.el9.x86_64 59/95 2026-02-25T14:22:02.028 INFO:teuthology.orchestra.run.trial189.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 60/95 2026-02-25T14:22:02.050 INFO:teuthology.orchestra.run.trial189.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 61/95 2026-02-25T14:22:02.304 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noar 87/95 2026-02-25T14:22:02.305 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7 88/95 2026-02-25T14:22:02.317 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7 88/95 2026-02-25T14:22:02.318 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noar 89/95 2026-02-25T14:22:02.375 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noar 89/95 2026-02-25T14:22:02.376 INFO:teuthology.orchestra.run.trial189.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 62/95 2026-02-25T14:22:02.418 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-mgr-modules-core-2:20.2.0-676.gc824a7fa.el9.n 90/95 2026-02-25T14:22:02.419 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 91/95 2026-02-25T14:22:02.441 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 91/95 2026-02-25T14:22:02.441 INFO:teuthology.orchestra.run.trial090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:22:02.441 INFO:teuthology.orchestra.run.trial090.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-25T14:22:02.442 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T14:22:02.442 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T14:22:02.442 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:02.450 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 63/95 2026-02-25T14:22:02.452 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 92/95 2026-02-25T14:22:02.675 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 92/95 2026-02-25T14:22:02.676 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 93/95 2026-02-25T14:22:02.699 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 93/95 2026-02-25T14:22:02.699 INFO:teuthology.orchestra.run.trial090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:22:02.699 INFO:teuthology.orchestra.run.trial090.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-25T14:22:02.699 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T14:22:02.699 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T14:22:02.699 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:02.795 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 94/95 2026-02-25T14:22:02.816 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 94/95 2026-02-25T14:22:02.816 INFO:teuthology.orchestra.run.trial090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:22:02.816 INFO:teuthology.orchestra.run.trial090.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-25T14:22:02.816 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T14:22:02.817 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T14:22:02.817 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:02.835 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 95/95 2026-02-25T14:22:03.113 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 64/95 2026-02-25T14:22:03.129 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 65/95 2026-02-25T14:22:03.139 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 66/95 2026-02-25T14:22:03.146 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/95 2026-02-25T14:22:03.159 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-ply-3.11-14.el9.noarch 68/95 2026-02-25T14:22:03.174 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 69/95 2026-02-25T14:22:03.248 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 70/95 2026-02-25T14:22:03.257 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 71/95 2026-02-25T14:22:03.280 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 72/95 2026-02-25T14:22:03.309 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 73/95 2026-02-25T14:22:03.358 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 74/95 2026-02-25T14:22:03.364 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 75/95 2026-02-25T14:22:03.368 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/95 2026-02-25T14:22:03.369 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 77/95 2026-02-25T14:22:03.386 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 78/95 2026-02-25T14:22:03.386 INFO:teuthology.orchestra.run.trial189.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-25T14:22:03.386 INFO:teuthology.orchestra.run.trial189.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-25T14:22:03.386 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:03.394 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 78/95 2026-02-25T14:22:03.419 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 78/95 2026-02-25T14:22:03.419 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-25T14:22:03.419 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:03.452 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 79/95 2026-02-25T14:22:03.516 INFO:teuthology.orchestra.run.trial189.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 80/95 2026-02-25T14:22:03.517 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 81/95 2026-02-25T14:22:03.531 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 81/95 2026-02-25T14:22:03.531 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:22:03.531 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-25T14:22:03.531 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:03.559 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 95/95 2026-02-25T14:22:03.560 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 1/95 2026-02-25T14:22:03.560 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 2/95 2026-02-25T14:22:03.560 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 3/95 2026-02-25T14:22:03.560 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 4/95 2026-02-25T14:22:03.560 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 5/95 2026-02-25T14:22:03.560 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-grafana-dashboards-2:20.2.0-676.gc824a7fa.el9 6/95 2026-02-25T14:22:03.560 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 7/95 2026-02-25T14:22:03.560 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noar 8/95 2026-02-25T14:22:03.561 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7 9/95 2026-02-25T14:22:03.561 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noar 10/95 2026-02-25T14:22:03.561 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-mgr-modules-core-2:20.2.0-676.gc824a7fa.el9.n 11/95 2026-02-25T14:22:03.561 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 12/95 2026-02-25T14:22:03.561 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-prometheus-alerts-2:20.2.0-676.gc824a7fa.el9. 13/95 2026-02-25T14:22:03.561 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 14/95 2026-02-25T14:22:03.561 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 15/95 2026-02-25T14:22:03.561 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 16/95 2026-02-25T14:22:03.561 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/95 2026-02-25T14:22:03.561 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/95 2026-02-25T14:22:03.561 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/95 2026-02-25T14:22:03.562 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/95 2026-02-25T14:22:03.562 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/95 2026-02-25T14:22:03.562 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/95 2026-02-25T14:22:03.562 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/95 2026-02-25T14:22:03.562 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/95 2026-02-25T14:22:03.562 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/95 2026-02-25T14:22:03.562 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/95 2026-02-25T14:22:03.562 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/95 2026-02-25T14:22:03.562 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/95 2026-02-25T14:22:03.562 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : zip-3.0-35.el9.x86_64 29/95 2026-02-25T14:22:03.562 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/95 2026-02-25T14:22:03.562 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/95 2026-02-25T14:22:03.563 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/95 2026-02-25T14:22:03.563 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/95 2026-02-25T14:22:03.563 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/95 2026-02-25T14:22:03.563 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/95 2026-02-25T14:22:03.563 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/95 2026-02-25T14:22:03.563 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/95 2026-02-25T14:22:03.563 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/95 2026-02-25T14:22:03.563 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 39/95 2026-02-25T14:22:03.563 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/95 2026-02-25T14:22:03.563 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 41/95 2026-02-25T14:22:03.564 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 42/95 2026-02-25T14:22:03.564 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 43/95 2026-02-25T14:22:03.564 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/95 2026-02-25T14:22:03.564 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/95 2026-02-25T14:22:03.564 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/95 2026-02-25T14:22:03.564 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/95 2026-02-25T14:22:03.564 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/95 2026-02-25T14:22:03.564 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 49/95 2026-02-25T14:22:03.564 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 50/95 2026-02-25T14:22:03.564 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/95 2026-02-25T14:22:03.564 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/95 2026-02-25T14:22:03.564 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/95 2026-02-25T14:22:03.565 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/95 2026-02-25T14:22:03.565 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/95 2026-02-25T14:22:03.565 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/95 2026-02-25T14:22:03.565 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/95 2026-02-25T14:22:03.565 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/95 2026-02-25T14:22:03.565 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/95 2026-02-25T14:22:03.565 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/95 2026-02-25T14:22:03.565 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/95 2026-02-25T14:22:03.565 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/95 2026-02-25T14:22:03.565 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/95 2026-02-25T14:22:03.565 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/95 2026-02-25T14:22:03.565 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/95 2026-02-25T14:22:03.565 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/95 2026-02-25T14:22:03.566 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/95 2026-02-25T14:22:03.566 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/95 2026-02-25T14:22:03.566 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/95 2026-02-25T14:22:03.566 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/95 2026-02-25T14:22:03.566 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/95 2026-02-25T14:22:03.566 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/95 2026-02-25T14:22:03.566 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/95 2026-02-25T14:22:03.566 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/95 2026-02-25T14:22:03.566 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/95 2026-02-25T14:22:03.566 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/95 2026-02-25T14:22:03.566 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/95 2026-02-25T14:22:03.566 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/95 2026-02-25T14:22:03.567 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/95 2026-02-25T14:22:03.567 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/95 2026-02-25T14:22:03.567 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 81/95 2026-02-25T14:22:03.567 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 82/95 2026-02-25T14:22:03.567 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 83/95 2026-02-25T14:22:03.567 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 84/95 2026-02-25T14:22:03.567 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 85/95 2026-02-25T14:22:03.567 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 86/95 2026-02-25T14:22:03.567 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 87/95 2026-02-25T14:22:03.567 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 88/95 2026-02-25T14:22:03.567 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 89/95 2026-02-25T14:22:03.567 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 90/95 2026-02-25T14:22:03.567 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 91/95 2026-02-25T14:22:03.567 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 92/95 2026-02-25T14:22:03.568 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 93/95 2026-02-25T14:22:03.568 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 94/95 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 95/95 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: ceph-grafana-dashboards-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr-modules-core-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: ceph-prometheus-alerts-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-25T14:22:03.632 INFO:teuthology.orchestra.run.trial090.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: lua-5.4.4-4.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-25T14:22:03.633 INFO:teuthology.orchestra.run.trial090.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-ply-3.11-14.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: unzip-6.0-59.el9.x86_64 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: zip-3.0-35.el9.x86_64 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:03.634 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:03.686 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-base 2026-02-25T14:22:04.002 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:30 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:04.067 INFO:teuthology.orchestra.run.trial090.stdout:Package ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:04.085 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:04.086 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:22:04.086 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:04.107 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install cephadm 2026-02-25T14:22:04.203 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 82/95 2026-02-25T14:22:04.228 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 82/95 2026-02-25T14:22:04.229 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:22:04.229 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-25T14:22:04.229 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T14:22:04.229 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T14:22:04.229 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:04.287 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 83/95 2026-02-25T14:22:04.289 INFO:teuthology.orchestra.run.trial189.stdout: Installing : cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 83/95 2026-02-25T14:22:04.296 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-prometheus-alerts-2:20.2.0-676.gc824a7fa.el9. 84/95 2026-02-25T14:22:04.319 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-grafana-dashboards-2:20.2.0-676.gc824a7fa.el9 85/95 2026-02-25T14:22:04.320 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 86/95 2026-02-25T14:22:04.428 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:30 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:04.493 INFO:teuthology.orchestra.run.trial090.stdout:Package cephadm-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:22:04.512 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:04.512 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:22:04.512 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:04.535 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T14:22:04.858 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:30 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:04.942 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:04.942 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:04.942 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:22:04.942 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:04.942 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:22:04.942 INFO:teuthology.orchestra.run.trial090.stdout: ceph-immutable-object-cache x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 154 k 2026-02-25T14:22:04.942 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:04.942 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:22:04.942 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:04.942 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:22:04.942 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:04.943 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 154 k 2026-02-25T14:22:04.943 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 447 k 2026-02-25T14:22:04.943 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:22:05.403 INFO:teuthology.orchestra.run.trial090.stdout:ceph-immutable-object-cache-20.2.0-676.gc824a7f 335 kB/s | 154 kB 00:00 2026-02-25T14:22:05.403 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:05.404 INFO:teuthology.orchestra.run.trial090.stdout:Total 334 kB/s | 154 kB 00:00 2026-02-25T14:22:05.404 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:22:05.409 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:22:05.409 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:22:05.440 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:22:05.440 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:22:05.462 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 86/95 2026-02-25T14:22:05.467 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noar 87/95 2026-02-25T14:22:05.496 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:22:05.498 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.e 1/1 2026-02-25T14:22:05.521 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.e 1/1 2026-02-25T14:22:05.521 INFO:teuthology.orchestra.run.trial090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:22:05.521 INFO:teuthology.orchestra.run.trial090.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-25T14:22:05.521 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:05.849 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.e 1/1 2026-02-25T14:22:05.849 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:05.850 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:22:05.850 INFO:teuthology.orchestra.run.trial090.stdout: ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:05.850 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:05.850 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:05.878 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-mgr 2026-02-25T14:22:05.922 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noar 87/95 2026-02-25T14:22:05.923 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7 88/95 2026-02-25T14:22:05.935 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7 88/95 2026-02-25T14:22:05.935 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noar 89/95 2026-02-25T14:22:05.989 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noar 89/95 2026-02-25T14:22:06.031 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-modules-core-2:20.2.0-676.gc824a7fa.el9.n 90/95 2026-02-25T14:22:06.032 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 91/95 2026-02-25T14:22:06.053 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 91/95 2026-02-25T14:22:06.053 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:22:06.053 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-25T14:22:06.053 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T14:22:06.054 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T14:22:06.054 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:06.064 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 92/95 2026-02-25T14:22:06.196 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:32 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:06.262 INFO:teuthology.orchestra.run.trial090.stdout:Package ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:06.280 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:06.281 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:22:06.281 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:06.286 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 92/95 2026-02-25T14:22:06.288 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 93/95 2026-02-25T14:22:06.303 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T14:22:06.309 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 93/95 2026-02-25T14:22:06.309 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:22:06.309 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-25T14:22:06.309 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T14:22:06.309 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T14:22:06.309 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:06.406 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 94/95 2026-02-25T14:22:06.427 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 94/95 2026-02-25T14:22:06.427 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:22:06.427 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-25T14:22:06.427 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T14:22:06.427 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T14:22:06.428 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:06.448 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 95/95 2026-02-25T14:22:06.622 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:32 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:06.686 INFO:teuthology.orchestra.run.trial090.stdout:Package ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:22:06.705 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:06.705 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:22:06.705 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:06.726 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T14:22:07.043 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:33 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:07.109 INFO:teuthology.orchestra.run.trial090.stdout:Package ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:22:07.127 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:07.128 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:22:07.128 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:07.150 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-mgr-rook 2026-02-25T14:22:07.160 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 95/95 2026-02-25T14:22:07.160 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 1/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 2/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 3/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 4/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 5/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-grafana-dashboards-2:20.2.0-676.gc824a7fa.el9 6/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 7/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noar 8/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7 9/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noar 10/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-modules-core-2:20.2.0-676.gc824a7fa.el9.n 11/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 12/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-prometheus-alerts-2:20.2.0-676.gc824a7fa.el9. 13/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 14/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 15/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 16/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : zip-3.0-35.el9.x86_64 29/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/95 2026-02-25T14:22:07.161 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 39/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 41/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 42/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 43/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 49/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 50/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/95 2026-02-25T14:22:07.162 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 81/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 82/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 83/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 84/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 85/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 86/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 87/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 88/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 89/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 90/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 91/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 92/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 93/95 2026-02-25T14:22:07.163 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 94/95 2026-02-25T14:22:07.234 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 95/95 2026-02-25T14:22:07.235 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:07.235 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:22:07.235 INFO:teuthology.orchestra.run.trial189.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-25T14:22:07.235 INFO:teuthology.orchestra.run.trial189.stdout: ceph-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:07.235 INFO:teuthology.orchestra.run.trial189.stdout: ceph-grafana-dashboards-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:07.235 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mds-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:07.235 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:07.236 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:07.236 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:07.236 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:07.236 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-k8sevents-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:07.236 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-modules-core-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:07.236 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:07.236 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mon-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:07.236 INFO:teuthology.orchestra.run.trial189.stdout: ceph-osd-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:07.236 INFO:teuthology.orchestra.run.trial189.stdout: ceph-prometheus-alerts-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:07.236 INFO:teuthology.orchestra.run.trial189.stdout: ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:07.237 INFO:teuthology.orchestra.run.trial189.stdout: cephadm-2:20.2.0-676.gc824a7fa.el9.noarch 2026-02-25T14:22:07.237 INFO:teuthology.orchestra.run.trial189.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-25T14:22:07.237 INFO:teuthology.orchestra.run.trial189.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-25T14:22:07.237 INFO:teuthology.orchestra.run.trial189.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-25T14:22:07.237 INFO:teuthology.orchestra.run.trial189.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-25T14:22:07.237 INFO:teuthology.orchestra.run.trial189.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-25T14:22:07.237 INFO:teuthology.orchestra.run.trial189.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-25T14:22:07.237 INFO:teuthology.orchestra.run.trial189.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-25T14:22:07.237 INFO:teuthology.orchestra.run.trial189.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-25T14:22:07.237 INFO:teuthology.orchestra.run.trial189.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-25T14:22:07.238 INFO:teuthology.orchestra.run.trial189.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T14:22:07.238 INFO:teuthology.orchestra.run.trial189.stdout: lua-5.4.4-4.el9.x86_64 2026-02-25T14:22:07.238 INFO:teuthology.orchestra.run.trial189.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-25T14:22:07.238 INFO:teuthology.orchestra.run.trial189.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-25T14:22:07.238 INFO:teuthology.orchestra.run.trial189.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-25T14:22:07.238 INFO:teuthology.orchestra.run.trial189.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-25T14:22:07.238 INFO:teuthology.orchestra.run.trial189.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-25T14:22:07.238 INFO:teuthology.orchestra.run.trial189.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-25T14:22:07.238 INFO:teuthology.orchestra.run.trial189.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-25T14:22:07.238 INFO:teuthology.orchestra.run.trial189.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-25T14:22:07.239 INFO:teuthology.orchestra.run.trial189.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-25T14:22:07.239 INFO:teuthology.orchestra.run.trial189.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-25T14:22:07.239 INFO:teuthology.orchestra.run.trial189.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-25T14:22:07.239 INFO:teuthology.orchestra.run.trial189.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-25T14:22:07.239 INFO:teuthology.orchestra.run.trial189.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-25T14:22:07.239 INFO:teuthology.orchestra.run.trial189.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-25T14:22:07.239 INFO:teuthology.orchestra.run.trial189.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-25T14:22:07.239 INFO:teuthology.orchestra.run.trial189.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-25T14:22:07.239 INFO:teuthology.orchestra.run.trial189.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-25T14:22:07.239 INFO:teuthology.orchestra.run.trial189.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-25T14:22:07.240 INFO:teuthology.orchestra.run.trial189.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-25T14:22:07.240 INFO:teuthology.orchestra.run.trial189.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-25T14:22:07.240 INFO:teuthology.orchestra.run.trial189.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-25T14:22:07.240 INFO:teuthology.orchestra.run.trial189.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-25T14:22:07.240 INFO:teuthology.orchestra.run.trial189.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-25T14:22:07.240 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-25T14:22:07.240 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-25T14:22:07.240 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-25T14:22:07.240 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-25T14:22:07.240 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-25T14:22:07.240 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-25T14:22:07.241 INFO:teuthology.orchestra.run.trial189.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-25T14:22:07.241 INFO:teuthology.orchestra.run.trial189.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-25T14:22:07.241 INFO:teuthology.orchestra.run.trial189.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-25T14:22:07.241 INFO:teuthology.orchestra.run.trial189.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T14:22:07.241 INFO:teuthology.orchestra.run.trial189.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-25T14:22:07.241 INFO:teuthology.orchestra.run.trial189.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-25T14:22:07.241 INFO:teuthology.orchestra.run.trial189.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-25T14:22:07.241 INFO:teuthology.orchestra.run.trial189.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-25T14:22:07.241 INFO:teuthology.orchestra.run.trial189.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-25T14:22:07.242 INFO:teuthology.orchestra.run.trial189.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-25T14:22:07.242 INFO:teuthology.orchestra.run.trial189.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-25T14:22:07.242 INFO:teuthology.orchestra.run.trial189.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-25T14:22:07.242 INFO:teuthology.orchestra.run.trial189.stdout: python3-ply-3.11-14.el9.noarch 2026-02-25T14:22:07.242 INFO:teuthology.orchestra.run.trial189.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-25T14:22:07.242 INFO:teuthology.orchestra.run.trial189.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-25T14:22:07.242 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-25T14:22:07.242 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-25T14:22:07.242 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-25T14:22:07.242 INFO:teuthology.orchestra.run.trial189.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-25T14:22:07.243 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-25T14:22:07.243 INFO:teuthology.orchestra.run.trial189.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-25T14:22:07.243 INFO:teuthology.orchestra.run.trial189.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-25T14:22:07.243 INFO:teuthology.orchestra.run.trial189.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-25T14:22:07.243 INFO:teuthology.orchestra.run.trial189.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-25T14:22:07.243 INFO:teuthology.orchestra.run.trial189.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-25T14:22:07.243 INFO:teuthology.orchestra.run.trial189.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-25T14:22:07.243 INFO:teuthology.orchestra.run.trial189.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-25T14:22:07.243 INFO:teuthology.orchestra.run.trial189.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-25T14:22:07.243 INFO:teuthology.orchestra.run.trial189.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-25T14:22:07.243 INFO:teuthology.orchestra.run.trial189.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-25T14:22:07.244 INFO:teuthology.orchestra.run.trial189.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-25T14:22:07.244 INFO:teuthology.orchestra.run.trial189.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-25T14:22:07.244 INFO:teuthology.orchestra.run.trial189.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-25T14:22:07.244 INFO:teuthology.orchestra.run.trial189.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-25T14:22:07.244 INFO:teuthology.orchestra.run.trial189.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-25T14:22:07.244 INFO:teuthology.orchestra.run.trial189.stdout: unzip-6.0-59.el9.x86_64 2026-02-25T14:22:07.244 INFO:teuthology.orchestra.run.trial189.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-25T14:22:07.244 INFO:teuthology.orchestra.run.trial189.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-25T14:22:07.244 INFO:teuthology.orchestra.run.trial189.stdout: zip-3.0-35.el9.x86_64 2026-02-25T14:22:07.244 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:07.244 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:07.290 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-base 2026-02-25T14:22:07.471 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:33 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:07.536 INFO:teuthology.orchestra.run.trial090.stdout:Package ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:22:07.554 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:07.555 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:22:07.555 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:07.576 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T14:22:07.609 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:33 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:07.675 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-base-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:07.693 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:07.694 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:22:07.694 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:07.716 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install cephadm 2026-02-25T14:22:07.895 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:33 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:07.961 INFO:teuthology.orchestra.run.trial090.stdout:Package ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:22:07.980 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:07.981 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:22:07.981 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:08.003 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-fuse 2026-02-25T14:22:08.031 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:34 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:08.097 INFO:teuthology.orchestra.run.trial189.stdout:Package cephadm-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:22:08.115 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:08.117 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:22:08.117 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:08.139 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T14:22:08.321 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:34 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:08.406 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:08.407 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:08.407 INFO:teuthology.orchestra.run.trial090.stdout: Package Architecture Version Repository Size 2026-02-25T14:22:08.408 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:08.408 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:22:08.408 INFO:teuthology.orchestra.run.trial090.stdout: ceph-fuse x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 942 k 2026-02-25T14:22:08.408 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:08.408 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:22:08.408 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:08.408 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:22:08.408 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:08.408 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 942 k 2026-02-25T14:22:08.408 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 2.7 M 2026-02-25T14:22:08.408 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:22:08.454 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:34 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:08.537 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:08.538 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:08.538 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:22:08.538 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:08.538 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:22:08.538 INFO:teuthology.orchestra.run.trial189.stdout: ceph-immutable-object-cache x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 154 k 2026-02-25T14:22:08.538 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:08.538 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:22:08.538 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:08.538 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:22:08.538 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:08.538 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 154 k 2026-02-25T14:22:08.538 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 447 k 2026-02-25T14:22:08.538 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:22:09.130 INFO:teuthology.orchestra.run.trial090.stdout:ceph-fuse-20.2.0-676.gc824a7fa.el9.x86_64.rpm 1.3 MB/s | 942 kB 00:00 2026-02-25T14:22:09.130 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:09.130 INFO:teuthology.orchestra.run.trial090.stdout:Total 1.3 MB/s | 942 kB 00:00 2026-02-25T14:22:09.130 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:22:09.132 INFO:teuthology.orchestra.run.trial189.stdout:ceph-immutable-object-cache-20.2.0-676.gc824a7f 260 kB/s | 154 kB 00:00 2026-02-25T14:22:09.133 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:09.133 INFO:teuthology.orchestra.run.trial189.stdout:Total 259 kB/s | 154 kB 00:00 2026-02-25T14:22:09.133 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:22:09.135 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:22:09.135 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:22:09.137 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:22:09.138 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:22:09.168 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:22:09.169 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:22:09.170 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:22:09.170 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:22:09.227 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:22:09.228 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.e 1/1 2026-02-25T14:22:09.253 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.e 1/1 2026-02-25T14:22:09.253 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:22:09.253 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-25T14:22:09.253 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:09.259 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:22:09.277 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:09.522 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:09.577 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.e 1/1 2026-02-25T14:22:09.577 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:09.578 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:22:09.578 INFO:teuthology.orchestra.run.trial189.stdout: ceph-immutable-object-cache-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:09.578 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:09.578 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:09.586 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:09.586 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:09.586 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:22:09.586 INFO:teuthology.orchestra.run.trial090.stdout: ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:09.586 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:09.586 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:09.604 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mgr 2026-02-25T14:22:09.615 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-volume 2026-02-25T14:22:09.916 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:35 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:09.938 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:35 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:09.981 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-mgr-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:10.000 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:10.001 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:22:10.001 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:10.004 INFO:teuthology.orchestra.run.trial090.stdout:Package ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:22:10.021 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T14:22:10.022 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:10.023 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:22:10.023 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:10.045 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install librados-devel 2026-02-25T14:22:10.335 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:36 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:10.369 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:36 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:10.400 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-mgr-dashboard-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:22:10.418 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:10.419 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:22:10.419 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:10.440 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T14:22:10.454 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:10.454 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:10.454 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:22:10.454 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:10.454 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:22:10.454 INFO:teuthology.orchestra.run.trial090.stdout: librados-devel x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 126 k 2026-02-25T14:22:10.455 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:10.455 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:22:10.455 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:10.455 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:22:10.455 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:10.455 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 126 k 2026-02-25T14:22:10.455 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 449 k 2026-02-25T14:22:10.455 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:22:10.763 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:36 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:10.828 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-mgr-diskprediction-local-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:22:10.847 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:10.847 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:22:10.848 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:10.869 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mgr-rook 2026-02-25T14:22:10.934 INFO:teuthology.orchestra.run.trial090.stdout:librados-devel-20.2.0-676.gc824a7fa.el9.x86_64. 262 kB/s | 126 kB 00:00 2026-02-25T14:22:10.935 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:10.935 INFO:teuthology.orchestra.run.trial090.stdout:Total 261 kB/s | 126 kB 00:00 2026-02-25T14:22:10.935 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:22:10.938 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:22:10.938 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:22:10.971 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:22:10.971 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:22:11.026 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:22:11.043 INFO:teuthology.orchestra.run.trial090.stdout: Installing : librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:11.137 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:11.190 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:37 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:11.200 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:11.200 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:11.201 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:22:11.201 INFO:teuthology.orchestra.run.trial090.stdout: librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:11.201 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:11.201 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:11.228 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install libcephfs2 2026-02-25T14:22:11.255 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-mgr-rook-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:22:11.274 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:11.274 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:22:11.274 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:11.296 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T14:22:11.550 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:37 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:11.612 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:37 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:11.617 INFO:teuthology.orchestra.run.trial090.stdout:Package libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:11.636 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:11.636 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:22:11.636 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:11.657 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install libcephfs-devel 2026-02-25T14:22:11.680 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-mgr-cephadm-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:22:11.699 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:11.699 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:22:11.699 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:11.720 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-fuse 2026-02-25T14:22:11.982 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:37 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:12.039 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:38 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:12.067 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:12.068 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:12.068 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:22:12.068 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:12.068 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:22:12.068 INFO:teuthology.orchestra.run.trial090.stdout: libcephfs-devel x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 34 k 2026-02-25T14:22:12.068 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:12.068 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:22:12.068 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:12.069 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:22:12.069 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:12.069 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 34 k 2026-02-25T14:22:12.069 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 155 k 2026-02-25T14:22:12.069 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:22:12.125 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:12.125 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:12.125 INFO:teuthology.orchestra.run.trial189.stdout: Package Architecture Version Repository Size 2026-02-25T14:22:12.125 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:12.125 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:22:12.126 INFO:teuthology.orchestra.run.trial189.stdout: ceph-fuse x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 942 k 2026-02-25T14:22:12.126 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:12.126 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:22:12.126 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:12.126 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:22:12.126 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:12.126 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 942 k 2026-02-25T14:22:12.126 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 2.7 M 2026-02-25T14:22:12.126 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:22:12.402 INFO:teuthology.orchestra.run.trial090.stdout:libcephfs-devel-20.2.0-676.gc824a7fa.el9.x86_64 103 kB/s | 34 kB 00:00 2026-02-25T14:22:12.403 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:12.403 INFO:teuthology.orchestra.run.trial090.stdout:Total 102 kB/s | 34 kB 00:00 2026-02-25T14:22:12.403 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:22:12.405 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:22:12.405 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:22:12.409 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:22:12.409 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:22:12.430 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:22:12.448 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:12.507 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:12.571 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:12.571 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:12.571 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:22:12.571 INFO:teuthology.orchestra.run.trial090.stdout: libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:12.571 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:12.571 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:12.600 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install librados2 2026-02-25T14:22:12.918 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:38 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:12.984 INFO:teuthology.orchestra.run.trial090.stdout:Package librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:12.987 INFO:teuthology.orchestra.run.trial189.stdout:ceph-fuse-20.2.0-676.gc824a7fa.el9.x86_64.rpm 1.1 MB/s | 942 kB 00:00 2026-02-25T14:22:12.988 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:12.988 INFO:teuthology.orchestra.run.trial189.stdout:Total 1.1 MB/s | 942 kB 00:00 2026-02-25T14:22:12.988 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:22:12.993 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:22:12.993 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:22:13.002 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:13.003 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:22:13.003 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:13.025 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install librbd1 2026-02-25T14:22:13.027 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:22:13.027 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:22:13.117 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:22:13.134 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:13.344 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:39 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:13.387 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:13.409 INFO:teuthology.orchestra.run.trial090.stdout:Package librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:13.428 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:13.429 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:22:13.429 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:13.449 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:13.449 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:13.449 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:22:13.450 INFO:teuthology.orchestra.run.trial189.stdout: ceph-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:13.450 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:13.450 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:13.451 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install python3-rados 2026-02-25T14:22:13.478 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-volume 2026-02-25T14:22:13.770 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:39 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:13.787 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:39 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:13.835 INFO:teuthology.orchestra.run.trial090.stdout:Package python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:13.852 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-volume-2:20.2.0-676.gc824a7fa.el9.noarch is already installed. 2026-02-25T14:22:13.854 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:13.854 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:22:13.855 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:13.870 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:13.871 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:22:13.871 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:13.876 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install python3-rgw 2026-02-25T14:22:13.892 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install librados-devel 2026-02-25T14:22:14.195 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:40 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:14.209 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:40 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:14.261 INFO:teuthology.orchestra.run.trial090.stdout:Package python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:14.279 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:14.280 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:22:14.280 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:14.295 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:14.295 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:14.295 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:22:14.295 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:14.295 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:22:14.295 INFO:teuthology.orchestra.run.trial189.stdout: librados-devel x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 126 k 2026-02-25T14:22:14.296 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:14.296 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:22:14.296 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:14.296 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:22:14.296 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:14.296 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 126 k 2026-02-25T14:22:14.296 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 449 k 2026-02-25T14:22:14.296 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:22:14.302 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install python3-cephfs 2026-02-25T14:22:14.621 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:40 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:14.687 INFO:teuthology.orchestra.run.trial090.stdout:Package python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:14.706 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:14.707 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:22:14.707 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:14.728 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install python3-rbd 2026-02-25T14:22:14.764 INFO:teuthology.orchestra.run.trial189.stdout:librados-devel-20.2.0-676.gc824a7fa.el9.x86_64. 268 kB/s | 126 kB 00:00 2026-02-25T14:22:14.765 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:14.765 INFO:teuthology.orchestra.run.trial189.stdout:Total 267 kB/s | 126 kB 00:00 2026-02-25T14:22:14.765 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:22:14.768 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:22:14.768 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:22:14.801 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:22:14.801 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:22:14.856 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:22:14.873 INFO:teuthology.orchestra.run.trial189.stdout: Installing : librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:14.965 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:15.028 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:15.028 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:15.028 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:22:15.028 INFO:teuthology.orchestra.run.trial189.stdout: librados-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:15.028 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:15.028 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:15.045 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:41 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:15.055 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install libcephfs2 2026-02-25T14:22:15.112 INFO:teuthology.orchestra.run.trial090.stdout:Package python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:15.131 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:15.132 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:22:15.132 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:15.153 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install rbd-fuse 2026-02-25T14:22:15.380 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:41 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:15.448 INFO:teuthology.orchestra.run.trial189.stdout:Package libcephfs2-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:15.467 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:15.467 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:22:15.468 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:15.468 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:41 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:15.489 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install libcephfs-devel 2026-02-25T14:22:15.553 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:15.553 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:15.553 INFO:teuthology.orchestra.run.trial090.stdout: Package Architecture Version Repository Size 2026-02-25T14:22:15.553 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:15.553 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:22:15.553 INFO:teuthology.orchestra.run.trial090.stdout: rbd-fuse x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 91 k 2026-02-25T14:22:15.553 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:15.554 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:22:15.554 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:15.554 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:22:15.554 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:15.554 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 91 k 2026-02-25T14:22:15.554 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 238 k 2026-02-25T14:22:15.554 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:22:15.810 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:41 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:15.895 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:15.895 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:15.895 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:22:15.895 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:15.895 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:22:15.895 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs-devel x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 34 k 2026-02-25T14:22:15.895 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:15.895 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:22:15.895 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:15.895 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:22:15.895 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:15.895 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 34 k 2026-02-25T14:22:15.896 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 155 k 2026-02-25T14:22:15.896 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:22:15.976 INFO:teuthology.orchestra.run.trial090.stdout:rbd-fuse-20.2.0-676.gc824a7fa.el9.x86_64.rpm 215 kB/s | 91 kB 00:00 2026-02-25T14:22:15.977 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:15.977 INFO:teuthology.orchestra.run.trial090.stdout:Total 215 kB/s | 91 kB 00:00 2026-02-25T14:22:15.977 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:22:15.982 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:22:15.982 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:22:16.014 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:22:16.014 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:22:16.065 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:22:16.083 INFO:teuthology.orchestra.run.trial090.stdout: Installing : rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:16.169 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:16.207 INFO:teuthology.orchestra.run.trial189.stdout:libcephfs-devel-20.2.0-676.gc824a7fa.el9.x86_64 110 kB/s | 34 kB 00:00 2026-02-25T14:22:16.208 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:16.209 INFO:teuthology.orchestra.run.trial189.stdout:Total 110 kB/s | 34 kB 00:00 2026-02-25T14:22:16.209 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:22:16.211 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:22:16.212 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:22:16.214 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:22:16.215 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:22:16.233 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:16.233 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:16.233 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:22:16.234 INFO:teuthology.orchestra.run.trial090.stdout: rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:16.234 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:16.234 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:16.237 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:22:16.254 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:16.263 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install rbd-mirror 2026-02-25T14:22:16.318 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:16.380 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:16.380 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:16.380 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:22:16.380 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs-devel-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:16.381 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:16.381 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:16.408 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install librados2 2026-02-25T14:22:16.584 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:42 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:16.670 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:16.670 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:16.670 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:22:16.670 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:16.670 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:22:16.670 INFO:teuthology.orchestra.run.trial090.stdout: rbd-mirror x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.9 M 2026-02-25T14:22:16.671 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:16.671 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:22:16.671 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:16.671 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:22:16.671 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:16.671 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 2.9 M 2026-02-25T14:22:16.671 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 11 M 2026-02-25T14:22:16.671 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:22:16.728 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:42 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:16.795 INFO:teuthology.orchestra.run.trial189.stdout:Package librados2-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:16.813 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:16.814 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:22:16.814 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:16.835 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install librbd1 2026-02-25T14:22:17.027 INFO:teuthology.orchestra.run.trial093.stdout:(2/2): ceph-base-debuginfo-20.2.0-676.gc824a7fa 2.9 MB/s | 107 MB 00:36 2026-02-25T14:22:17.028 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:17.028 INFO:teuthology.orchestra.run.trial093.stdout:Total 3.2 MB/s | 118 MB 00:36 2026-02-25T14:22:17.028 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:22:17.033 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:22:17.033 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:22:17.152 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:43 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:17.219 INFO:teuthology.orchestra.run.trial189.stdout:Package librbd1-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:17.238 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:17.238 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:22:17.239 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:17.260 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-rados 2026-02-25T14:22:17.452 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:22:17.452 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:22:17.566 INFO:teuthology.orchestra.run.trial090.stdout:rbd-mirror-20.2.0-676.gc824a7fa.el9.x86_64.rpm 3.3 MB/s | 2.9 MB 00:00 2026-02-25T14:22:17.566 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:17.566 INFO:teuthology.orchestra.run.trial090.stdout:Total 3.3 MB/s | 2.9 MB 00:00 2026-02-25T14:22:17.566 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:22:17.571 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:22:17.571 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:22:17.581 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:43 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:17.611 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:22:17.612 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:22:17.647 INFO:teuthology.orchestra.run.trial189.stdout:Package python3-rados-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:17.665 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:17.666 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:22:17.666 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:17.687 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-rgw 2026-02-25T14:22:17.793 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:22:17.794 INFO:teuthology.orchestra.run.trial090.stdout: Installing : rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:17.817 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:17.817 INFO:teuthology.orchestra.run.trial090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:22:17.817 INFO:teuthology.orchestra.run.trial090.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-25T14:22:17.817 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T14:22:17.817 INFO:teuthology.orchestra.run.trial090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T14:22:17.817 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:18.002 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:44 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:18.069 INFO:teuthology.orchestra.run.trial189.stdout:Package python3-rgw-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:18.087 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:18.088 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:22:18.088 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:18.109 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-cephfs 2026-02-25T14:22:18.152 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:18.152 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:18.152 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:22:18.153 INFO:teuthology.orchestra.run.trial090.stdout: rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:18.153 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:18.153 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:18.181 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install rbd-nbd 2026-02-25T14:22:18.423 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:44 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:18.488 INFO:teuthology.orchestra.run.trial189.stdout:Package python3-cephfs-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:18.507 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:18.507 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:22:18.508 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:18.509 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:44 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:18.528 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-rbd 2026-02-25T14:22:18.595 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:18.595 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:18.595 INFO:teuthology.orchestra.run.trial090.stdout: Package Architecture Version Repository Size 2026-02-25T14:22:18.595 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:18.595 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:22:18.595 INFO:teuthology.orchestra.run.trial090.stdout: rbd-nbd x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 180 k 2026-02-25T14:22:18.595 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:18.595 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:22:18.595 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:18.595 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:22:18.595 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:18.596 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 180 k 2026-02-25T14:22:18.596 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 498 k 2026-02-25T14:22:18.596 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:22:18.848 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:44 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:18.915 INFO:teuthology.orchestra.run.trial189.stdout:Package python3-rbd-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:22:18.934 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:18.935 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:22:18.935 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:18.957 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install rbd-fuse 2026-02-25T14:22:19.041 INFO:teuthology.orchestra.run.trial090.stdout:rbd-nbd-20.2.0-676.gc824a7fa.el9.x86_64.rpm 404 kB/s | 180 kB 00:00 2026-02-25T14:22:19.041 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:19.042 INFO:teuthology.orchestra.run.trial090.stdout:Total 403 kB/s | 180 kB 00:00 2026-02-25T14:22:19.042 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:22:19.046 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:22:19.047 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:22:19.078 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:22:19.078 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:22:19.135 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:22:19.152 INFO:teuthology.orchestra.run.trial090.stdout: Installing : rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:19.244 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:19.268 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:45 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:19.308 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:19.308 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:19.308 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:22:19.308 INFO:teuthology.orchestra.run.trial090.stdout: rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:19.308 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:19.308 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:22:19.340 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-base-debuginfo 2026-02-25T14:22:19.351 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:19.351 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:19.351 INFO:teuthology.orchestra.run.trial189.stdout: Package Architecture Version Repository Size 2026-02-25T14:22:19.351 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:19.351 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:22:19.351 INFO:teuthology.orchestra.run.trial189.stdout: rbd-fuse x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 91 k 2026-02-25T14:22:19.351 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:19.351 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:22:19.351 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:19.351 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:22:19.351 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:19.351 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 91 k 2026-02-25T14:22:19.351 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 238 k 2026-02-25T14:22:19.351 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:22:19.657 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:01:45 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:19.723 INFO:teuthology.orchestra.run.trial189.stdout:rbd-fuse-20.2.0-676.gc824a7fa.el9.x86_64.rpm 245 kB/s | 91 kB 00:00 2026-02-25T14:22:19.724 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:19.724 INFO:teuthology.orchestra.run.trial189.stdout:Total 245 kB/s | 91 kB 00:00 2026-02-25T14:22:19.725 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:22:19.729 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:22:19.729 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:22:19.742 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:22:19.743 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:19.743 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:22:19.743 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:19.743 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:22:19.743 INFO:teuthology.orchestra.run.trial090.stdout: ceph-base-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 107 M 2026-02-25T14:22:19.743 INFO:teuthology.orchestra.run.trial090.stdout:Installing weak dependencies: 2026-02-25T14:22:19.743 INFO:teuthology.orchestra.run.trial090.stdout: ceph-debugsource x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 11 M 2026-02-25T14:22:19.743 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:19.743 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:22:19.743 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:22:19.743 INFO:teuthology.orchestra.run.trial090.stdout:Install 2 Packages 2026-02-25T14:22:19.743 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:22:19.743 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 118 M 2026-02-25T14:22:19.743 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 751 M 2026-02-25T14:22:19.743 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:22:19.760 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:22:19.760 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:22:19.812 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:22:19.829 INFO:teuthology.orchestra.run.trial189.stdout: Installing : rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:19.915 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:19.946 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:22:19.978 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:19.978 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:19.978 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:22:19.978 INFO:teuthology.orchestra.run.trial189.stdout: rbd-fuse-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:19.979 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:19.979 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:20.006 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install rbd-mirror 2026-02-25T14:22:20.327 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:46 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:20.411 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:20.411 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:20.412 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:22:20.412 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:20.412 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:22:20.412 INFO:teuthology.orchestra.run.trial189.stdout: rbd-mirror x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.9 M 2026-02-25T14:22:20.412 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:20.412 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:22:20.412 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:20.412 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:22:20.412 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:20.412 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 2.9 M 2026-02-25T14:22:20.413 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 11 M 2026-02-25T14:22:20.413 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:22:21.267 INFO:teuthology.orchestra.run.trial189.stdout:rbd-mirror-20.2.0-676.gc824a7fa.el9.x86_64.rpm 3.4 MB/s | 2.9 MB 00:00 2026-02-25T14:22:21.267 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:21.267 INFO:teuthology.orchestra.run.trial189.stdout:Total 3.4 MB/s | 2.9 MB 00:00 2026-02-25T14:22:21.267 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:22:21.273 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:22:21.274 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:22:21.314 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:22:21.314 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:22:21.495 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:22:21.496 INFO:teuthology.orchestra.run.trial189.stdout: Installing : rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:21.519 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:21.519 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T14:22:21.519 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-25T14:22:21.519 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T14:22:21.520 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T14:22:21.520 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:21.818 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:21.818 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:21.818 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:22:21.818 INFO:teuthology.orchestra.run.trial189.stdout: rbd-mirror-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:21.818 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:21.818 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:21.846 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install rbd-nbd 2026-02-25T14:22:22.163 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:48 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:22.247 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:22.248 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:22.248 INFO:teuthology.orchestra.run.trial189.stdout: Package Architecture Version Repository Size 2026-02-25T14:22:22.248 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:22.248 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:22:22.248 INFO:teuthology.orchestra.run.trial189.stdout: rbd-nbd x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 180 k 2026-02-25T14:22:22.248 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:22.248 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:22:22.248 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:22.249 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:22:22.249 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:22.249 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 180 k 2026-02-25T14:22:22.249 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 498 k 2026-02-25T14:22:22.249 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:22:22.376 INFO:teuthology.orchestra.run.trial090.stdout:(1/2): ceph-debugsource-20.2.0-676.gc824a7fa.el 4.2 MB/s | 11 MB 00:02 2026-02-25T14:22:22.620 INFO:teuthology.orchestra.run.trial189.stdout:rbd-nbd-20.2.0-676.gc824a7fa.el9.x86_64.rpm 483 kB/s | 180 kB 00:00 2026-02-25T14:22:22.620 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:22.620 INFO:teuthology.orchestra.run.trial189.stdout:Total 482 kB/s | 180 kB 00:00 2026-02-25T14:22:22.621 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:22:22.625 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:22:22.625 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:22:22.657 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:22:22.657 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:22:22.718 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:22:22.735 INFO:teuthology.orchestra.run.trial189.stdout: Installing : rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:22.825 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:22.888 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:22:22.888 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:22.888 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:22:22.888 INFO:teuthology.orchestra.run.trial189.stdout: rbd-nbd-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:22.888 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:22.888 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:22:22.916 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-base-debuginfo 2026-02-25T14:22:23.239 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:49 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:23.325 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:22:23.325 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:23.325 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:22:23.326 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:23.326 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:22:23.326 INFO:teuthology.orchestra.run.trial189.stdout: ceph-base-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 107 M 2026-02-25T14:22:23.326 INFO:teuthology.orchestra.run.trial189.stdout:Installing weak dependencies: 2026-02-25T14:22:23.326 INFO:teuthology.orchestra.run.trial189.stdout: ceph-debugsource x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 11 M 2026-02-25T14:22:23.326 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:23.326 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:22:23.326 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:22:23.326 INFO:teuthology.orchestra.run.trial189.stdout:Install 2 Packages 2026-02-25T14:22:23.326 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:22:23.326 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 118 M 2026-02-25T14:22:23.326 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 751 M 2026-02-25T14:22:23.326 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:22:23.451 INFO:teuthology.orchestra.run.trial094.stdout:(2/2): ceph-base-debuginfo-20.2.0-676.gc824a7fa 2.2 MB/s | 107 MB 00:47 2026-02-25T14:22:23.453 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:22:23.453 INFO:teuthology.orchestra.run.trial094.stdout:Total 2.5 MB/s | 118 MB 00:47 2026-02-25T14:22:23.453 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:22:23.458 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:22:23.458 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:22:23.872 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:22:23.872 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:22:25.395 INFO:teuthology.orchestra.run.trial189.stdout:(1/2): ceph-debugsource-20.2.0-676.gc824a7fa.el 5.3 MB/s | 11 MB 00:02 2026-02-25T14:22:25.504 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-debugsource-2:20.2.0-676.gc824a7fa.el9.x86_64 1/2 2026-02-25T14:22:25.523 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 2/2 2026-02-25T14:22:26.157 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 2/2 2026-02-25T14:22:26.158 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/2 2026-02-25T14:22:26.221 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-debugsource-2:20.2.0-676.gc824a7fa.el9.x86_64 2/2 2026-02-25T14:22:26.221 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:22:26.221 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:22:26.221 INFO:teuthology.orchestra.run.trial093.stdout: ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:26.221 INFO:teuthology.orchestra.run.trial093.stdout: ceph-debugsource-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:26.221 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:22:26.221 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:22:26.263 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-common-debuginfo 2026-02-25T14:22:26.369 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:22:26.587 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:01:52 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:26.672 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:22:26.673 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:22:26.673 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:22:26.673 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:22:26.673 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:22:26.673 INFO:teuthology.orchestra.run.trial093.stdout: ceph-common-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 586 M 2026-02-25T14:22:26.673 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:22:26.673 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:22:26.673 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:22:26.673 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:22:26.673 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:22:26.674 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 586 M 2026-02-25T14:22:26.674 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 3.5 G 2026-02-25T14:22:26.674 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:22:32.016 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-debugsource-2:20.2.0-676.gc824a7fa.el9.x86_64 1/2 2026-02-25T14:22:32.035 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 2/2 2026-02-25T14:22:32.608 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 2/2 2026-02-25T14:22:32.609 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/2 2026-02-25T14:22:32.673 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-debugsource-2:20.2.0-676.gc824a7fa.el9.x86_64 2/2 2026-02-25T14:22:32.673 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:22:32.674 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:22:32.674 INFO:teuthology.orchestra.run.trial094.stdout: ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:32.674 INFO:teuthology.orchestra.run.trial094.stdout: ceph-debugsource-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:22:32.674 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:22:32.674 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:22:32.715 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-common-debuginfo 2026-02-25T14:22:33.036 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:01:59 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:22:33.121 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:22:33.121 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:22:33.121 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:22:33.121 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:22:33.121 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:22:33.121 INFO:teuthology.orchestra.run.trial094.stdout: ceph-common-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 586 M 2026-02-25T14:22:33.121 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:22:33.121 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:22:33.121 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:22:33.121 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:22:33.121 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:22:33.121 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 586 M 2026-02-25T14:22:33.121 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 3.5 G 2026-02-25T14:22:33.121 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:23:07.094 INFO:teuthology.orchestra.run.trial090.stdout:(2/2): ceph-base-debuginfo-20.2.0-676.gc824a7fa 2.3 MB/s | 107 MB 00:47 2026-02-25T14:23:07.095 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:23:07.095 INFO:teuthology.orchestra.run.trial090.stdout:Total 2.5 MB/s | 118 MB 00:47 2026-02-25T14:23:07.095 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:23:07.100 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:23:07.100 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:23:07.514 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:23:07.514 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:23:09.991 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:23:15.537 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-debugsource-2:20.2.0-676.gc824a7fa.el9.x86_64 1/2 2026-02-25T14:23:15.558 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 2/2 2026-02-25T14:23:16.144 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 2/2 2026-02-25T14:23:16.144 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/2 2026-02-25T14:23:16.208 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-debugsource-2:20.2.0-676.gc824a7fa.el9.x86_64 2/2 2026-02-25T14:23:16.208 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:23:16.208 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:23:16.208 INFO:teuthology.orchestra.run.trial090.stdout: ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:23:16.208 INFO:teuthology.orchestra.run.trial090.stdout: ceph-debugsource-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:23:16.208 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:23:16.208 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:23:16.248 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-common-debuginfo 2026-02-25T14:23:16.573 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:02:42 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:23:16.658 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:23:16.658 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:23:16.658 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:23:16.658 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:23:16.658 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:23:16.658 INFO:teuthology.orchestra.run.trial090.stdout: ceph-common-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 586 M 2026-02-25T14:23:16.659 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:23:16.659 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:23:16.659 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:23:16.659 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:23:16.659 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:23:16.659 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 586 M 2026-02-25T14:23:16.659 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 3.5 G 2026-02-25T14:23:16.659 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:23:24.421 INFO:teuthology.orchestra.run.trial189.stdout:(2/2): ceph-base-debuginfo-20.2.0-676.gc824a7fa 1.7 MB/s | 107 MB 01:01 2026-02-25T14:23:24.422 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:23:24.423 INFO:teuthology.orchestra.run.trial189.stdout:Total 1.9 MB/s | 118 MB 01:01 2026-02-25T14:23:24.423 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:23:24.428 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:23:24.428 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:23:24.843 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:23:24.843 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:23:27.362 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:23:32.910 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-debugsource-2:20.2.0-676.gc824a7fa.el9.x86_64 1/2 2026-02-25T14:23:32.930 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 2/2 2026-02-25T14:23:33.536 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 2/2 2026-02-25T14:23:33.536 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/2 2026-02-25T14:23:33.599 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-debugsource-2:20.2.0-676.gc824a7fa.el9.x86_64 2/2 2026-02-25T14:23:33.599 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:23:33.599 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:23:33.599 INFO:teuthology.orchestra.run.trial189.stdout: ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:23:33.600 INFO:teuthology.orchestra.run.trial189.stdout: ceph-debugsource-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:23:33.600 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:23:33.600 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:23:33.641 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-common-debuginfo 2026-02-25T14:23:33.968 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:02:59 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:23:34.052 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:23:34.052 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:23:34.053 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:23:34.053 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:23:34.053 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:23:34.053 INFO:teuthology.orchestra.run.trial189.stdout: ceph-common-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 586 M 2026-02-25T14:23:34.053 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:23:34.053 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:23:34.053 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:23:34.053 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:23:34.053 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:23:34.053 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 586 M 2026-02-25T14:23:34.053 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 3.5 G 2026-02-25T14:23:34.054 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:26:31.404 INFO:teuthology.orchestra.run.trial090.stdout:ceph-common-debuginfo-20.2.0-676.gc824a7fa.el9. 3.0 MB/s | 586 MB 03:14 2026-02-25T14:26:31.404 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:26:31.404 INFO:teuthology.orchestra.run.trial090.stdout:Total 3.0 MB/s | 586 MB 03:14 2026-02-25T14:26:31.405 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:26:31.407 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:26:31.407 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:26:33.152 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:26:33.152 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:26:46.338 INFO:teuthology.orchestra.run.trial094.stdout:ceph-common-debuginfo-20.2.0-676.gc824a7fa.el9. 2.3 MB/s | 586 MB 04:13 2026-02-25T14:26:46.339 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:26:46.339 INFO:teuthology.orchestra.run.trial094.stdout:Total 2.3 MB/s | 586 MB 04:13 2026-02-25T14:26:46.339 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:26:46.341 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:26:46.341 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:26:48.080 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:26:48.081 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:26:58.987 INFO:teuthology.orchestra.run.trial093.stdout:ceph-common-debuginfo-20.2.0-676.gc824a7fa.el9. 2.2 MB/s | 586 MB 04:32 2026-02-25T14:26:58.987 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:26:58.988 INFO:teuthology.orchestra.run.trial093.stdout:Total 2.2 MB/s | 586 MB 04:32 2026-02-25T14:26:58.988 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:26:58.990 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:26:58.990 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:27:00.732 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:27:00.732 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:27:06.640 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:27:06.660 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:27:09.101 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:27:09.165 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:27:09.165 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:27:09.166 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:27:09.166 INFO:teuthology.orchestra.run.trial090.stdout: ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:27:09.166 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:27:09.166 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:27:09.254 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-25T14:27:09.584 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:06:35 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:27:09.670 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:27:09.670 INFO:teuthology.orchestra.run.trial090.stdout:======================================================================================= 2026-02-25T14:27:09.670 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:27:09.670 INFO:teuthology.orchestra.run.trial090.stdout:======================================================================================= 2026-02-25T14:27:09.670 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:27:09.671 INFO:teuthology.orchestra.run.trial090.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 3.0 M 2026-02-25T14:27:09.671 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:27:09.671 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:27:09.671 INFO:teuthology.orchestra.run.trial090.stdout:======================================================================================= 2026-02-25T14:27:09.671 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:27:09.671 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:27:09.671 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 3.0 M 2026-02-25T14:27:09.671 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 15 M 2026-02-25T14:27:09.671 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:27:10.893 INFO:teuthology.orchestra.run.trial090.stdout:ceph-immutable-object-cache-debuginfo-20.2.0-67 2.5 MB/s | 3.0 MB 00:01 2026-02-25T14:27:10.893 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:27:10.893 INFO:teuthology.orchestra.run.trial090.stdout:Total 2.5 MB/s | 3.0 MB 00:01 2026-02-25T14:27:10.893 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:27:10.895 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:27:10.895 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:27:10.922 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:27:10.922 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:27:11.115 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:27:11.133 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:20.2.0-676.g 1/1 2026-02-25T14:27:11.206 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:20.2.0-676.g 1/1 2026-02-25T14:27:11.271 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:20.2.0-676.g 1/1 2026-02-25T14:27:11.271 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:27:11.271 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:27:11.271 INFO:teuthology.orchestra.run.trial090.stdout: ceph-immutable-object-cache-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:27:11.271 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:27:11.271 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:27:11.300 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-25T14:27:11.616 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:06:37 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:27:11.698 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:27:11.699 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:27:11.699 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:27:11.699 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:27:11.699 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:27:11.699 INFO:teuthology.orchestra.run.trial090.stdout: ceph-radosgw-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 543 M 2026-02-25T14:27:11.699 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:27:11.699 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:27:11.699 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:27:11.699 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:27:11.699 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:27:11.699 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 543 M 2026-02-25T14:27:11.699 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 4.1 G 2026-02-25T14:27:11.699 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:27:19.957 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:27:19.976 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:27:20.983 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:27:21.046 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:27:21.046 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:27:21.046 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:27:21.046 INFO:teuthology.orchestra.run.trial094.stdout: ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:27:21.046 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:27:21.046 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:27:21.129 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-25T14:27:21.446 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:06:47 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:27:21.529 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:27:21.529 INFO:teuthology.orchestra.run.trial094.stdout:======================================================================================= 2026-02-25T14:27:21.529 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:27:21.529 INFO:teuthology.orchestra.run.trial094.stdout:======================================================================================= 2026-02-25T14:27:21.529 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:27:21.530 INFO:teuthology.orchestra.run.trial094.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 3.0 M 2026-02-25T14:27:21.530 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:27:21.530 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:27:21.530 INFO:teuthology.orchestra.run.trial094.stdout:======================================================================================= 2026-02-25T14:27:21.530 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:27:21.530 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:27:21.530 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 3.0 M 2026-02-25T14:27:21.530 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 15 M 2026-02-25T14:27:21.530 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:27:22.858 INFO:teuthology.orchestra.run.trial094.stdout:ceph-immutable-object-cache-debuginfo-20.2.0-67 2.3 MB/s | 3.0 MB 00:01 2026-02-25T14:27:22.859 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:27:22.859 INFO:teuthology.orchestra.run.trial094.stdout:Total 2.3 MB/s | 3.0 MB 00:01 2026-02-25T14:27:22.859 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:27:22.860 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:27:22.860 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:27:22.887 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:27:22.887 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:27:23.075 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:27:23.094 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:20.2.0-676.g 1/1 2026-02-25T14:27:23.167 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:20.2.0-676.g 1/1 2026-02-25T14:27:23.231 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:20.2.0-676.g 1/1 2026-02-25T14:27:23.231 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:27:23.231 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:27:23.231 INFO:teuthology.orchestra.run.trial094.stdout: ceph-immutable-object-cache-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:27:23.231 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:27:23.231 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:27:23.259 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-25T14:27:23.580 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:06:49 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:27:23.666 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:27:23.667 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:27:23.667 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:27:23.667 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:27:23.667 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:27:23.667 INFO:teuthology.orchestra.run.trial094.stdout: ceph-radosgw-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 543 M 2026-02-25T14:27:23.667 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:27:23.667 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:27:23.667 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:27:23.667 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:27:23.668 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:27:23.668 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 543 M 2026-02-25T14:27:23.668 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 4.1 G 2026-02-25T14:27:23.668 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:27:32.820 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:27:32.839 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:27:35.297 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:27:35.360 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:27:35.361 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:27:35.361 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:27:35.361 INFO:teuthology.orchestra.run.trial093.stdout: ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:27:35.361 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:27:35.361 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:27:35.445 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-25T14:27:35.765 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:07:01 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:27:35.851 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:27:35.851 INFO:teuthology.orchestra.run.trial093.stdout:======================================================================================= 2026-02-25T14:27:35.851 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:27:35.851 INFO:teuthology.orchestra.run.trial093.stdout:======================================================================================= 2026-02-25T14:27:35.851 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:27:35.851 INFO:teuthology.orchestra.run.trial093.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 3.0 M 2026-02-25T14:27:35.851 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:27:35.851 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:27:35.852 INFO:teuthology.orchestra.run.trial093.stdout:======================================================================================= 2026-02-25T14:27:35.852 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:27:35.852 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:27:35.852 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 3.0 M 2026-02-25T14:27:35.852 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 15 M 2026-02-25T14:27:35.852 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:27:36.960 INFO:teuthology.orchestra.run.trial093.stdout:ceph-immutable-object-cache-debuginfo-20.2.0-67 2.7 MB/s | 3.0 MB 00:01 2026-02-25T14:27:36.961 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:27:36.961 INFO:teuthology.orchestra.run.trial093.stdout:Total 2.7 MB/s | 3.0 MB 00:01 2026-02-25T14:27:36.961 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:27:36.962 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:27:36.962 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:27:36.989 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:27:36.989 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:27:37.177 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:27:37.196 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:20.2.0-676.g 1/1 2026-02-25T14:27:37.268 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:20.2.0-676.g 1/1 2026-02-25T14:27:37.331 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:20.2.0-676.g 1/1 2026-02-25T14:27:37.332 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:27:37.332 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:27:37.332 INFO:teuthology.orchestra.run.trial093.stdout: ceph-immutable-object-cache-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:27:37.332 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:27:37.332 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:27:37.358 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-25T14:27:37.424 INFO:teuthology.orchestra.run.trial189.stdout:ceph-common-debuginfo-20.2.0-676.gc824a7fa.el9. 2.4 MB/s | 586 MB 04:03 2026-02-25T14:27:37.425 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:27:37.425 INFO:teuthology.orchestra.run.trial189.stdout:Total 2.4 MB/s | 586 MB 04:03 2026-02-25T14:27:37.425 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:27:37.427 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:27:37.427 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:27:37.681 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:07:03 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:27:37.763 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:27:37.764 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:27:37.764 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:27:37.764 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:27:37.764 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:27:37.764 INFO:teuthology.orchestra.run.trial093.stdout: ceph-radosgw-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 543 M 2026-02-25T14:27:37.764 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:27:37.764 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:27:37.764 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:27:37.764 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:27:37.764 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:27:37.764 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 543 M 2026-02-25T14:27:37.764 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 4.1 G 2026-02-25T14:27:37.764 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:27:39.166 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:27:39.167 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:28:11.677 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:28:11.695 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:28:12.725 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:28:12.788 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:28:12.789 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:28:12.789 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:28:12.789 INFO:teuthology.orchestra.run.trial189.stdout: ceph-common-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:28:12.789 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:28:12.789 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:28:12.872 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-25T14:28:13.192 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:07:39 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:28:13.277 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:28:13.278 INFO:teuthology.orchestra.run.trial189.stdout:======================================================================================= 2026-02-25T14:28:13.278 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:28:13.278 INFO:teuthology.orchestra.run.trial189.stdout:======================================================================================= 2026-02-25T14:28:13.278 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:28:13.279 INFO:teuthology.orchestra.run.trial189.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 3.0 M 2026-02-25T14:28:13.279 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:28:13.279 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:28:13.279 INFO:teuthology.orchestra.run.trial189.stdout:======================================================================================= 2026-02-25T14:28:13.279 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:28:13.279 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:28:13.279 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 3.0 M 2026-02-25T14:28:13.279 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 15 M 2026-02-25T14:28:13.279 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:28:14.172 INFO:teuthology.orchestra.run.trial189.stdout:ceph-immutable-object-cache-debuginfo-20.2.0-67 3.4 MB/s | 3.0 MB 00:00 2026-02-25T14:28:14.172 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:28:14.172 INFO:teuthology.orchestra.run.trial189.stdout:Total 3.4 MB/s | 3.0 MB 00:00 2026-02-25T14:28:14.172 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:28:14.174 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:28:14.174 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:28:14.201 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:28:14.201 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:28:14.389 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:28:14.407 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:20.2.0-676.g 1/1 2026-02-25T14:28:14.479 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:20.2.0-676.g 1/1 2026-02-25T14:28:14.544 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:20.2.0-676.g 1/1 2026-02-25T14:28:14.544 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:28:14.544 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:28:14.544 INFO:teuthology.orchestra.run.trial189.stdout: ceph-immutable-object-cache-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:28:14.544 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:28:14.544 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:28:14.571 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-25T14:28:14.893 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:07:40 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:28:14.976 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:28:14.977 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:28:14.977 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:28:14.978 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:28:14.978 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:28:14.978 INFO:teuthology.orchestra.run.trial189.stdout: ceph-radosgw-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 543 M 2026-02-25T14:28:14.978 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:28:14.978 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:28:14.978 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:28:14.978 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:28:14.978 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:28:14.978 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 543 M 2026-02-25T14:28:14.978 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 4.1 G 2026-02-25T14:28:14.978 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:30:02.919 INFO:teuthology.orchestra.run.trial090.stdout:ceph-radosgw-debuginfo-20.2.0-676.gc824a7fa.el9 3.2 MB/s | 543 MB 02:51 2026-02-25T14:30:02.926 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:30:02.926 INFO:teuthology.orchestra.run.trial090.stdout:Total 3.2 MB/s | 543 MB 02:51 2026-02-25T14:30:02.926 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:30:02.927 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:30:02.927 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:30:04.532 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:30:04.533 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:30:37.382 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:30:37.401 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x8 1/1 2026-02-25T14:30:38.159 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x8 1/1 2026-02-25T14:30:38.224 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x8 1/1 2026-02-25T14:30:38.224 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:30:38.224 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:30:38.224 INFO:teuthology.orchestra.run.trial090.stdout: ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:30:38.224 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:30:38.224 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:30:38.308 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-test-debuginfo 2026-02-25T14:30:38.636 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:10:04 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:30:38.721 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:30:38.722 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:30:38.722 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:30:38.722 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:30:38.722 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:30:38.722 INFO:teuthology.orchestra.run.trial090.stdout: ceph-test-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 1.7 G 2026-02-25T14:30:38.722 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:30:38.723 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:30:38.723 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:30:38.723 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:30:38.723 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:30:38.723 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 1.7 G 2026-02-25T14:30:38.723 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 12 G 2026-02-25T14:30:38.723 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:30:54.597 INFO:teuthology.orchestra.run.trial094.stdout:ceph-radosgw-debuginfo-20.2.0-676.gc824a7fa.el9 2.6 MB/s | 543 MB 03:30 2026-02-25T14:30:54.598 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:30:54.598 INFO:teuthology.orchestra.run.trial094.stdout:Total 2.6 MB/s | 543 MB 03:30 2026-02-25T14:30:54.598 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:30:54.600 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:30:54.600 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:30:54.983 INFO:teuthology.orchestra.run.trial189.stdout:ceph-radosgw-debuginfo-20.2.0-676.gc824a7fa.el9 3.4 MB/s | 543 MB 02:40 2026-02-25T14:30:54.984 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:30:54.984 INFO:teuthology.orchestra.run.trial189.stdout:Total 3.4 MB/s | 543 MB 02:40 2026-02-25T14:30:54.984 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:30:54.986 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:30:54.986 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:30:56.206 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:30:56.207 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:30:56.598 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:30:56.599 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:31:28.551 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:31:28.571 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x8 1/1 2026-02-25T14:31:28.843 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:31:28.862 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x8 1/1 2026-02-25T14:31:30.037 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x8 1/1 2026-02-25T14:31:30.101 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x8 1/1 2026-02-25T14:31:30.101 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:31:30.101 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:31:30.101 INFO:teuthology.orchestra.run.trial094.stdout: ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:31:30.101 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:31:30.101 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:31:30.183 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-test-debuginfo 2026-02-25T14:31:30.348 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x8 1/1 2026-02-25T14:31:30.412 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x8 1/1 2026-02-25T14:31:30.412 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:31:30.412 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:31:30.412 INFO:teuthology.orchestra.run.trial189.stdout: ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:31:30.412 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:31:30.413 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:31:30.494 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-test-debuginfo 2026-02-25T14:31:30.505 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:10:56 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:31:30.588 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:31:30.589 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:31:30.589 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:31:30.589 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:31:30.589 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:31:30.589 INFO:teuthology.orchestra.run.trial094.stdout: ceph-test-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 1.7 G 2026-02-25T14:31:30.589 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:31:30.589 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:31:30.589 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:31:30.589 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:31:30.589 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:31:30.589 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 1.7 G 2026-02-25T14:31:30.590 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 12 G 2026-02-25T14:31:30.590 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:31:30.823 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:10:56 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:31:30.906 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:31:30.907 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:31:30.907 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:31:30.907 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:31:30.907 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:31:30.907 INFO:teuthology.orchestra.run.trial189.stdout: ceph-test-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 1.7 G 2026-02-25T14:31:30.907 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:31:30.907 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:31:30.907 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:31:30.907 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:31:30.907 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:31:30.907 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 1.7 G 2026-02-25T14:31:30.907 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 12 G 2026-02-25T14:31:30.907 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:33:38.551 INFO:teuthology.orchestra.run.trial093.stdout:ceph-radosgw-debuginfo-20.2.0-676.gc824a7fa.el9 1.5 MB/s | 543 MB 06:00 2026-02-25T14:33:38.558 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:33:38.558 INFO:teuthology.orchestra.run.trial093.stdout:Total 1.5 MB/s | 543 MB 06:00 2026-02-25T14:33:38.558 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:33:38.558 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:33:38.558 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:33:40.172 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:33:40.172 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:34:13.063 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:34:13.081 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x8 1/1 2026-02-25T14:34:14.552 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x8 1/1 2026-02-25T14:34:14.615 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x8 1/1 2026-02-25T14:34:14.616 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:34:14.616 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:34:14.616 INFO:teuthology.orchestra.run.trial093.stdout: ceph-radosgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:34:14.616 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:34:14.616 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:34:14.692 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-test-debuginfo 2026-02-25T14:34:15.016 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:13:41 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:34:15.100 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:34:15.100 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:34:15.100 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:34:15.100 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:34:15.101 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:34:15.101 INFO:teuthology.orchestra.run.trial093.stdout: ceph-test-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 1.7 G 2026-02-25T14:34:15.101 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:34:15.101 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:34:15.101 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:34:15.101 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:34:15.101 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:34:15.101 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 1.7 G 2026-02-25T14:34:15.101 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 12 G 2026-02-25T14:34:15.101 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:46:13.033 INFO:teuthology.orchestra.run.trial189.stdout:ceph-test-debuginfo-20.2.0-676.gc824a7fa.el9.x8 2.0 MB/s | 1.7 GB 14:42 2026-02-25T14:46:13.034 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:46:13.034 INFO:teuthology.orchestra.run.trial189.stdout:Total 2.0 MB/s | 1.7 GB 14:42 2026-02-25T14:46:13.034 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:46:13.037 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:46:13.037 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:46:18.241 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:46:18.241 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:46:25.606 INFO:teuthology.orchestra.run.trial094.stdout:ceph-test-debuginfo-20.2.0-676.gc824a7fa.el9.x8 2.0 MB/s | 1.7 GB 14:55 2026-02-25T14:46:25.607 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:46:25.607 INFO:teuthology.orchestra.run.trial094.stdout:Total 2.0 MB/s | 1.7 GB 14:55 2026-02-25T14:46:25.607 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:46:25.610 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:46:25.610 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:46:30.793 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:46:30.794 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:47:50.182 INFO:teuthology.orchestra.run.trial093.stdout:ceph-test-debuginfo-20.2.0-676.gc824a7fa.el9.x8 2.2 MB/s | 1.7 GB 13:35 2026-02-25T14:47:50.182 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:47:50.182 INFO:teuthology.orchestra.run.trial093.stdout:Total 2.2 MB/s | 1.7 GB 13:35 2026-02-25T14:47:50.182 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:47:50.185 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:47:50.185 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:47:55.386 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:47:55.386 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:48:00.331 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:48:00.352 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:48:03.483 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:48:03.547 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:48:03.547 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:48:03.547 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:48:03.547 INFO:teuthology.orchestra.run.trial189.stdout: ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:48:03.547 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:48:03.547 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:48:03.749 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-base-debuginfo 2026-02-25T14:48:04.075 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:27:30 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:48:04.142 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:48:04.159 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:48:04.160 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-25T14:48:04.160 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:48:04.182 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mgr-debuginfo 2026-02-25T14:48:04.501 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:27:30 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:48:04.585 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:48:04.585 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:48:04.585 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:48:04.586 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:48:04.586 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:48:04.586 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 22 M 2026-02-25T14:48:04.586 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:48:04.586 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:48:04.586 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:48:04.586 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:48:04.586 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:48:04.587 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 22 M 2026-02-25T14:48:04.587 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 146 M 2026-02-25T14:48:04.587 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:48:10.682 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:48:10.704 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:48:13.061 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:48:13.124 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:48:13.124 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:48:13.124 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:48:13.124 INFO:teuthology.orchestra.run.trial094.stdout: ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:48:13.124 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:48:13.124 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:48:13.325 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-base-debuginfo 2026-02-25T14:48:13.649 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:27:39 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:48:13.694 INFO:teuthology.orchestra.run.trial189.stdout:ceph-mgr-debuginfo-20.2.0-676.gc824a7fa.el9.x86 2.4 MB/s | 22 MB 00:09 2026-02-25T14:48:13.695 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:48:13.695 INFO:teuthology.orchestra.run.trial189.stdout:Total 2.4 MB/s | 22 MB 00:09 2026-02-25T14:48:13.695 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:48:13.698 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:48:13.698 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:48:13.715 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:48:13.733 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:48:13.733 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T14:48:13.733 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:48:13.754 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr-debuginfo 2026-02-25T14:48:13.779 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:48:13.779 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:48:14.067 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:27:40 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:48:14.150 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:48:14.151 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:48:14.151 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:48:14.151 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:48:14.151 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:48:14.151 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 22 M 2026-02-25T14:48:14.151 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:48:14.151 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:48:14.151 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:48:14.152 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:48:14.152 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:48:14.152 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 22 M 2026-02-25T14:48:14.152 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 146 M 2026-02-25T14:48:14.152 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:48:15.052 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:48:15.070 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:48:15.236 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:48:15.300 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:48:15.300 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:48:15.300 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:48:15.300 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:48:15.300 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:48:15.301 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:48:15.328 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mds-debuginfo 2026-02-25T14:48:15.650 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:27:41 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:48:15.735 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:48:15.735 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:48:15.735 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:48:15.735 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:48:15.736 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:48:15.736 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mds-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 54 M 2026-02-25T14:48:15.736 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:48:15.736 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:48:15.736 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:48:15.736 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:48:15.736 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:48:15.736 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 54 M 2026-02-25T14:48:15.736 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 221 M 2026-02-25T14:48:15.736 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:48:40.842 INFO:teuthology.orchestra.run.trial094.stdout:ceph-mgr-debuginfo-20.2.0-676.gc824a7fa.el9.x86 840 kB/s | 22 MB 00:26 2026-02-25T14:48:40.842 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:48:40.842 INFO:teuthology.orchestra.run.trial094.stdout:Total 839 kB/s | 22 MB 00:26 2026-02-25T14:48:40.843 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:48:40.844 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:48:40.844 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:48:40.927 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:48:40.927 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:48:42.205 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:48:42.223 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:48:42.389 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:48:42.453 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:48:42.453 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:48:42.453 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:48:42.453 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:48:42.453 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:48:42.453 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:48:42.480 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mds-debuginfo 2026-02-25T14:48:42.807 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:28:08 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:48:42.889 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:48:42.890 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:48:42.890 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:48:42.890 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:48:42.890 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:48:42.890 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mds-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 54 M 2026-02-25T14:48:42.890 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:48:42.890 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:48:42.890 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:48:42.890 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:48:42.890 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:48:42.890 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 54 M 2026-02-25T14:48:42.890 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 221 M 2026-02-25T14:48:42.890 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:48:57.598 INFO:teuthology.orchestra.run.trial189.stdout:ceph-mds-debuginfo-20.2.0-676.gc824a7fa.el9.x86 1.3 MB/s | 54 MB 00:41 2026-02-25T14:48:57.599 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:48:57.599 INFO:teuthology.orchestra.run.trial189.stdout:Total 1.3 MB/s | 54 MB 00:41 2026-02-25T14:48:57.599 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:48:57.601 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:48:57.601 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:48:57.777 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:48:57.777 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:49:00.425 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:49:00.443 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:49:00.667 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:49:00.732 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:49:00.732 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:49:00.732 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:49:00.732 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:49:00.733 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:49:00.733 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:49:00.764 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mon-debuginfo 2026-02-25T14:49:01.086 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:28:27 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:49:01.168 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:49:01.169 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:49:01.169 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:49:01.169 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:49:01.169 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:49:01.170 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mon-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 129 M 2026-02-25T14:49:01.170 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:49:01.170 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:49:01.170 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:49:01.170 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:49:01.170 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:49:01.170 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 129 M 2026-02-25T14:49:01.170 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 641 M 2026-02-25T14:49:01.170 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:49:11.997 INFO:teuthology.orchestra.run.trial094.stdout:ceph-mds-debuginfo-20.2.0-676.gc824a7fa.el9.x86 1.9 MB/s | 54 MB 00:29 2026-02-25T14:49:11.997 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:49:11.997 INFO:teuthology.orchestra.run.trial094.stdout:Total 1.9 MB/s | 54 MB 00:29 2026-02-25T14:49:11.997 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:49:11.999 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:49:11.999 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:49:12.176 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:49:12.176 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:49:14.826 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:49:14.844 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:49:15.067 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:49:15.135 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:49:15.135 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:49:15.135 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:49:15.135 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:49:15.135 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:49:15.135 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:49:15.164 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mon-debuginfo 2026-02-25T14:49:15.479 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:28:41 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:49:15.562 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:49:15.563 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:49:15.563 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:49:15.563 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:49:15.563 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:49:15.563 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mon-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 129 M 2026-02-25T14:49:15.563 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:49:15.564 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:49:15.564 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:49:15.564 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:49:15.564 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:49:15.564 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 129 M 2026-02-25T14:49:15.564 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 641 M 2026-02-25T14:49:15.564 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:49:35.396 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:49:35.417 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:49:38.525 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:49:38.589 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:49:38.589 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:49:38.590 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:49:38.590 INFO:teuthology.orchestra.run.trial093.stdout: ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:49:38.590 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:49:38.590 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:49:38.790 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-base-debuginfo 2026-02-25T14:49:39.115 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:29:05 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:49:39.180 INFO:teuthology.orchestra.run.trial093.stdout:Package ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:49:39.198 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:49:39.199 INFO:teuthology.orchestra.run.trial093.stdout:Nothing to do. 2026-02-25T14:49:39.199 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:49:39.221 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-mgr-debuginfo 2026-02-25T14:49:39.540 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:29:05 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:49:39.623 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:49:39.624 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:49:39.624 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:49:39.624 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:49:39.624 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:49:39.624 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 22 M 2026-02-25T14:49:39.624 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:49:39.624 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:49:39.624 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:49:39.624 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:49:39.625 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:49:39.625 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 22 M 2026-02-25T14:49:39.625 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 146 M 2026-02-25T14:49:39.625 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:49:51.520 INFO:teuthology.orchestra.run.trial093.stdout:ceph-mgr-debuginfo-20.2.0-676.gc824a7fa.el9.x86 1.8 MB/s | 22 MB 00:11 2026-02-25T14:49:51.521 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:49:51.521 INFO:teuthology.orchestra.run.trial093.stdout:Total 1.8 MB/s | 22 MB 00:11 2026-02-25T14:49:51.521 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:49:51.523 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:49:51.523 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:49:51.605 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:49:51.605 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:49:52.881 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:49:52.900 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:49:53.069 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:49:53.133 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:49:53.134 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:49:53.134 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:49:53.134 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:49:53.134 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:49:53.134 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:49:53.162 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-mds-debuginfo 2026-02-25T14:49:53.480 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:29:19 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:49:53.563 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:49:53.564 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:49:53.564 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:49:53.564 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:49:53.565 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:49:53.565 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mds-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 54 M 2026-02-25T14:49:53.565 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:49:53.565 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:49:53.565 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:49:53.565 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:49:53.565 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:49:53.565 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 54 M 2026-02-25T14:49:53.565 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 221 M 2026-02-25T14:49:53.565 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:50:19.238 INFO:teuthology.orchestra.run.trial093.stdout:ceph-mds-debuginfo-20.2.0-676.gc824a7fa.el9.x86 2.1 MB/s | 54 MB 00:25 2026-02-25T14:50:19.238 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:50:19.238 INFO:teuthology.orchestra.run.trial093.stdout:Total 2.1 MB/s | 54 MB 00:25 2026-02-25T14:50:19.239 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:50:19.241 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:50:19.241 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:50:19.416 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:50:19.416 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:50:21.172 INFO:teuthology.orchestra.run.trial189.stdout:ceph-mon-debuginfo-20.2.0-676.gc824a7fa.el9.x86 1.6 MB/s | 129 MB 01:20 2026-02-25T14:50:21.173 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:50:21.173 INFO:teuthology.orchestra.run.trial189.stdout:Total 1.6 MB/s | 129 MB 01:20 2026-02-25T14:50:21.173 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:50:21.175 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:50:21.175 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:50:21.573 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:50:21.574 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:50:22.126 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:50:22.144 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:50:22.368 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:50:22.434 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:50:22.434 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:50:22.434 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:50:22.434 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:50:22.434 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:50:22.435 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:50:22.465 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-mon-debuginfo 2026-02-25T14:50:22.787 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:29:48 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:50:22.870 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:50:22.871 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:50:22.871 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:50:22.871 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:50:22.871 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:50:22.871 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mon-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 129 M 2026-02-25T14:50:22.871 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:50:22.871 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:50:22.871 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:50:22.872 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:50:22.872 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:50:22.872 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 129 M 2026-02-25T14:50:22.872 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 641 M 2026-02-25T14:50:22.872 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:50:28.214 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:50:28.233 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:50:28.762 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:50:28.827 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:50:28.827 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:50:28.827 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:50:28.827 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:50:28.827 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:50:28.827 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:50:28.865 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-osd-debuginfo 2026-02-25T14:50:29.188 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:29:55 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:50:29.272 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:50:29.273 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:50:29.273 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:50:29.273 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:50:29.273 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:50:29.273 INFO:teuthology.orchestra.run.trial189.stdout: ceph-osd-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 343 M 2026-02-25T14:50:29.273 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:50:29.273 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:50:29.274 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:50:29.274 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:50:29.274 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:50:29.274 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 343 M 2026-02-25T14:50:29.274 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 1.6 G 2026-02-25T14:50:29.274 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:50:31.761 INFO:teuthology.orchestra.run.trial094.stdout:ceph-mon-debuginfo-20.2.0-676.gc824a7fa.el9.x86 1.7 MB/s | 129 MB 01:16 2026-02-25T14:50:31.761 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:50:31.761 INFO:teuthology.orchestra.run.trial094.stdout:Total 1.7 MB/s | 129 MB 01:16 2026-02-25T14:50:31.762 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:50:31.763 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:50:31.764 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:50:32.161 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:50:32.162 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:50:38.798 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:50:38.817 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:50:39.343 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:50:39.408 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:50:39.408 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:50:39.408 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:50:39.408 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:50:39.408 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:50:39.408 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:50:39.447 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-osd-debuginfo 2026-02-25T14:50:39.771 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:30:05 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:50:39.854 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:50:39.854 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:50:39.854 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:50:39.854 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:50:39.854 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:50:39.854 INFO:teuthology.orchestra.run.trial094.stdout: ceph-osd-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 343 M 2026-02-25T14:50:39.855 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:50:39.855 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:50:39.855 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:50:39.855 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:50:39.855 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:50:39.855 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 343 M 2026-02-25T14:50:39.855 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 1.6 G 2026-02-25T14:50:39.855 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:52:02.131 INFO:teuthology.orchestra.run.trial093.stdout:ceph-mon-debuginfo-20.2.0-676.gc824a7fa.el9.x86 1.3 MB/s | 129 MB 01:39 2026-02-25T14:52:02.138 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:52:02.138 INFO:teuthology.orchestra.run.trial093.stdout:Total 1.3 MB/s | 129 MB 01:39 2026-02-25T14:52:02.138 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:52:02.139 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:52:02.139 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:52:02.532 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:52:02.532 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:52:09.142 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:52:09.160 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:52:09.689 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:52:09.754 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:52:09.754 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:52:09.754 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:52:09.754 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:52:09.754 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:52:09.754 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:52:09.791 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-osd-debuginfo 2026-02-25T14:52:10.110 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:31:36 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:52:10.193 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:52:10.194 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:52:10.194 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:52:10.194 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:52:10.194 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:52:10.195 INFO:teuthology.orchestra.run.trial093.stdout: ceph-osd-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 343 M 2026-02-25T14:52:10.195 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:52:10.195 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:52:10.195 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:52:10.195 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:52:10.195 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:52:10.195 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 343 M 2026-02-25T14:52:10.195 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 1.6 G 2026-02-25T14:52:10.195 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:54:20.898 INFO:teuthology.orchestra.run.trial094.stdout:ceph-osd-debuginfo-20.2.0-676.gc824a7fa.el9.x86 1.6 MB/s | 343 MB 03:41 2026-02-25T14:54:20.899 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:54:20.899 INFO:teuthology.orchestra.run.trial094.stdout:Total 1.6 MB/s | 343 MB 03:41 2026-02-25T14:54:20.899 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:54:20.901 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:54:20.901 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:54:21.926 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:54:21.926 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:54:39.135 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:54:39.152 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:54:40.318 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:54:40.382 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:54:40.383 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:40.383 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:54:40.383 INFO:teuthology.orchestra.run.trial094.stdout: ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:54:40.383 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:40.383 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:54:40.440 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-fuse-debuginfo 2026-02-25T14:54:40.760 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:34:06 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:54:40.843 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:54:40.843 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:54:40.843 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:54:40.843 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:54:40.843 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:54:40.843 INFO:teuthology.orchestra.run.trial094.stdout: ceph-fuse-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 10 M 2026-02-25T14:54:40.843 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:40.843 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:54:40.843 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:54:40.843 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:54:40.843 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:40.844 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 10 M 2026-02-25T14:54:40.844 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 51 M 2026-02-25T14:54:40.844 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:54:44.362 INFO:teuthology.orchestra.run.trial094.stdout:ceph-fuse-debuginfo-20.2.0-676.gc824a7fa.el9.x8 2.9 MB/s | 10 MB 00:03 2026-02-25T14:54:44.363 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:54:44.363 INFO:teuthology.orchestra.run.trial094.stdout:Total 2.9 MB/s | 10 MB 00:03 2026-02-25T14:54:44.363 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:54:44.364 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:54:44.364 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:54:44.412 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:54:44.412 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:54:44.989 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:54:45.008 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:54:45.106 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:54:45.170 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:54:45.170 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:45.170 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:54:45.170 INFO:teuthology.orchestra.run.trial094.stdout: ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:54:45.170 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:45.171 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:54:45.199 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install librados-devel-debuginfo 2026-02-25T14:54:45.518 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:34:11 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:54:45.601 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:54:45.601 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:54:45.601 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:54:45.601 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:54:45.602 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:54:45.602 INFO:teuthology.orchestra.run.trial094.stdout: librados-devel-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 144 k 2026-02-25T14:54:45.602 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:45.602 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:54:45.602 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:54:45.602 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:54:45.602 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:45.602 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 144 k 2026-02-25T14:54:45.602 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 802 k 2026-02-25T14:54:45.602 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:54:45.890 INFO:teuthology.orchestra.run.trial094.stdout:librados-devel-debuginfo-20.2.0-676.gc824a7fa.e 496 kB/s | 144 kB 00:00 2026-02-25T14:54:45.892 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:54:45.892 INFO:teuthology.orchestra.run.trial094.stdout:Total 492 kB/s | 144 kB 00:00 2026-02-25T14:54:45.892 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:54:45.894 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:54:45.895 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:54:45.913 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:54:45.913 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:54:45.955 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:54:45.973 INFO:teuthology.orchestra.run.trial094.stdout: Installing : librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:54:46.032 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:54:46.095 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:54:46.095 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:46.096 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:54:46.096 INFO:teuthology.orchestra.run.trial094.stdout: librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:54:46.096 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:46.096 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:54:46.121 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install libcephfs2-debuginfo 2026-02-25T14:54:46.435 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:34:12 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:54:46.517 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:54:46.518 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:54:46.518 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:54:46.518 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:54:46.518 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:54:46.518 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs2-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 9.1 M 2026-02-25T14:54:46.518 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:46.518 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:54:46.519 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:54:46.519 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:54:46.519 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:46.519 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 9.1 M 2026-02-25T14:54:46.519 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 46 M 2026-02-25T14:54:46.519 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:54:47.113 INFO:teuthology.orchestra.run.trial094.stdout:libcephfs2-debuginfo-20.2.0-676.gc824a7fa.el9.x 15 MB/s | 9.1 MB 00:00 2026-02-25T14:54:47.114 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:54:47.114 INFO:teuthology.orchestra.run.trial094.stdout:Total 15 MB/s | 9.1 MB 00:00 2026-02-25T14:54:47.114 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:54:47.115 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:54:47.115 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:54:47.160 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:54:47.161 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:54:47.657 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:54:47.675 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:54:47.773 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:54:47.837 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:54:47.837 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:47.837 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:54:47.837 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:54:47.837 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:47.837 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:54:47.865 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install librados2-debuginfo 2026-02-25T14:54:48.177 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:34:14 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:54:48.260 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:54:48.260 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:54:48.260 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:54:48.260 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:54:48.260 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:54:48.260 INFO:teuthology.orchestra.run.trial094.stdout: librados2-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 63 M 2026-02-25T14:54:48.260 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:48.260 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:54:48.260 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:54:48.260 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:54:48.260 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:48.261 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 63 M 2026-02-25T14:54:48.261 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 397 M 2026-02-25T14:54:48.261 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:54:48.291 INFO:teuthology.orchestra.run.trial090.stdout:ceph-test-debuginfo-20.2.0-676.gc824a7fa.el9.x8 1.2 MB/s | 1.7 GB 24:09 2026-02-25T14:54:48.291 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:54:48.291 INFO:teuthology.orchestra.run.trial090.stdout:Total 1.2 MB/s | 1.7 GB 24:09 2026-02-25T14:54:48.291 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:54:48.295 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:54:48.295 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:54:53.454 INFO:teuthology.orchestra.run.trial094.stdout:librados2-debuginfo-20.2.0-676.gc824a7fa.el9.x8 12 MB/s | 63 MB 00:05 2026-02-25T14:54:53.455 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:54:53.455 INFO:teuthology.orchestra.run.trial094.stdout:Total 12 MB/s | 63 MB 00:05 2026-02-25T14:54:53.455 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:54:53.457 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:54:53.457 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:54:53.489 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:54:53.489 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:54:53.663 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:54:53.664 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:54:57.169 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:54:57.189 INFO:teuthology.orchestra.run.trial094.stdout: Installing : librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:54:57.542 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:54:57.605 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:54:57.605 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:57.606 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:54:57.606 INFO:teuthology.orchestra.run.trial094.stdout: librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:54:57.606 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:57.606 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:54:57.637 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install librbd1-debuginfo 2026-02-25T14:54:57.897 INFO:teuthology.orchestra.run.trial189.stdout:ceph-osd-debuginfo-20.2.0-676.gc824a7fa.el9.x86 1.3 MB/s | 343 MB 04:28 2026-02-25T14:54:57.897 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:54:57.898 INFO:teuthology.orchestra.run.trial189.stdout:Total 1.3 MB/s | 343 MB 04:28 2026-02-25T14:54:57.898 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:54:57.899 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:54:57.899 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:54:57.953 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:34:23 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:54:58.037 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:54:58.037 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:54:58.037 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:54:58.038 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:54:58.038 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:54:58.038 INFO:teuthology.orchestra.run.trial094.stdout: librbd1-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 80 M 2026-02-25T14:54:58.038 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:58.038 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:54:58.038 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:54:58.038 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:54:58.038 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:54:58.038 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 80 M 2026-02-25T14:54:58.038 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 406 M 2026-02-25T14:54:58.039 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:54:58.737 INFO:teuthology.orchestra.run.trial093.stdout:ceph-osd-debuginfo-20.2.0-676.gc824a7fa.el9.x86 2.0 MB/s | 343 MB 02:48 2026-02-25T14:54:58.738 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:54:58.738 INFO:teuthology.orchestra.run.trial093.stdout:Total 2.0 MB/s | 343 MB 02:48 2026-02-25T14:54:58.738 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:54:58.740 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:54:58.740 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:54:58.922 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:54:58.922 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:54:59.765 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:54:59.765 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:55:16.450 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:55:16.468 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:55:16.977 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:55:16.995 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:55:17.652 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:55:17.716 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:55:17.716 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:17.716 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:55:17.716 INFO:teuthology.orchestra.run.trial189.stdout: ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:55:17.716 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:17.717 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:55:17.774 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-fuse-debuginfo 2026-02-25T14:55:18.093 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:34:44 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:55:18.177 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:55:18.178 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:55:18.178 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:55:18.178 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:55:18.178 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:55:18.178 INFO:teuthology.orchestra.run.trial189.stdout: ceph-fuse-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 10 M 2026-02-25T14:55:18.178 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:18.179 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:55:18.179 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:55:18.179 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:55:18.179 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:18.179 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 10 M 2026-02-25T14:55:18.179 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 51 M 2026-02-25T14:55:18.179 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:55:18.190 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:55:18.252 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:55:18.253 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:18.253 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:55:18.253 INFO:teuthology.orchestra.run.trial093.stdout: ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:55:18.253 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:18.253 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:55:18.308 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install ceph-fuse-debuginfo 2026-02-25T14:55:18.624 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:34:44 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:55:18.707 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:55:18.708 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:55:18.708 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:55:18.708 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:55:18.708 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:55:18.708 INFO:teuthology.orchestra.run.trial093.stdout: ceph-fuse-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 10 M 2026-02-25T14:55:18.708 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:18.708 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:55:18.709 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:55:18.709 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:55:18.709 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:18.709 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 10 M 2026-02-25T14:55:18.709 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 51 M 2026-02-25T14:55:18.709 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:55:22.311 INFO:teuthology.orchestra.run.trial189.stdout:ceph-fuse-debuginfo-20.2.0-676.gc824a7fa.el9.x8 2.5 MB/s | 10 MB 00:04 2026-02-25T14:55:22.312 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:55:22.312 INFO:teuthology.orchestra.run.trial189.stdout:Total 2.5 MB/s | 10 MB 00:04 2026-02-25T14:55:22.312 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:55:22.314 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:55:22.314 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:55:22.361 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:55:22.361 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:55:22.914 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:55:22.932 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:55:23.031 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:55:23.095 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:55:23.095 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:23.095 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:55:23.095 INFO:teuthology.orchestra.run.trial189.stdout: ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:55:23.095 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:23.095 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:55:23.122 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install librados-devel-debuginfo 2026-02-25T14:55:23.436 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:34:49 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:55:23.522 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:55:23.523 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:55:23.523 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:55:23.523 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:55:23.523 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:55:23.523 INFO:teuthology.orchestra.run.trial189.stdout: librados-devel-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 144 k 2026-02-25T14:55:23.523 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:23.523 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:55:23.523 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:55:23.523 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:55:23.523 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:23.524 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 144 k 2026-02-25T14:55:23.524 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 802 k 2026-02-25T14:55:23.524 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:55:24.107 INFO:teuthology.orchestra.run.trial189.stdout:librados-devel-debuginfo-20.2.0-676.gc824a7fa.e 245 kB/s | 144 kB 00:00 2026-02-25T14:55:24.108 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:55:24.108 INFO:teuthology.orchestra.run.trial189.stdout:Total 245 kB/s | 144 kB 00:00 2026-02-25T14:55:24.109 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:55:24.110 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:55:24.110 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:55:24.130 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:55:24.130 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:55:24.171 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:55:24.189 INFO:teuthology.orchestra.run.trial189.stdout: Installing : librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:55:24.251 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:55:24.315 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:55:24.315 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:24.315 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:55:24.315 INFO:teuthology.orchestra.run.trial189.stdout: librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:55:24.315 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:24.316 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:55:24.343 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install libcephfs2-debuginfo 2026-02-25T14:55:24.659 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:34:50 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:55:24.742 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:55:24.742 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:55:24.743 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:55:24.743 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:55:24.743 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:55:24.743 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs2-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 9.1 M 2026-02-25T14:55:24.743 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:24.743 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:55:24.743 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:55:24.743 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:55:24.743 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:24.743 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 9.1 M 2026-02-25T14:55:24.743 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 46 M 2026-02-25T14:55:24.743 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:55:29.017 INFO:teuthology.orchestra.run.trial093.stdout:ceph-fuse-debuginfo-20.2.0-676.gc824a7fa.el9.x8 1.0 MB/s | 10 MB 00:10 2026-02-25T14:55:29.018 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:55:29.018 INFO:teuthology.orchestra.run.trial093.stdout:Total 1.0 MB/s | 10 MB 00:10 2026-02-25T14:55:29.018 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:55:29.020 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:55:29.020 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:55:29.067 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:55:29.067 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:55:29.122 INFO:teuthology.orchestra.run.trial189.stdout:libcephfs2-debuginfo-20.2.0-676.gc824a7fa.el9.x 2.1 MB/s | 9.1 MB 00:04 2026-02-25T14:55:29.123 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:55:29.123 INFO:teuthology.orchestra.run.trial189.stdout:Total 2.1 MB/s | 9.1 MB 00:04 2026-02-25T14:55:29.123 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:55:29.125 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:55:29.125 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:55:29.168 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:55:29.169 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:55:29.628 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:55:29.646 INFO:teuthology.orchestra.run.trial093.stdout: Installing : ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:55:29.663 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:55:29.681 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:55:29.743 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:55:29.775 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:55:29.807 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:55:29.807 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:29.807 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:55:29.807 INFO:teuthology.orchestra.run.trial093.stdout: ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:55:29.808 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:29.808 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:55:29.836 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install librados-devel-debuginfo 2026-02-25T14:55:29.839 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:55:29.839 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:29.839 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:55:29.839 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:55:29.839 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:29.839 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:55:29.866 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install librados2-debuginfo 2026-02-25T14:55:30.153 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:34:56 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:55:30.177 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:34:56 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:55:30.236 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:55:30.237 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:55:30.237 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:55:30.237 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:55:30.237 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:55:30.237 INFO:teuthology.orchestra.run.trial093.stdout: librados-devel-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 144 k 2026-02-25T14:55:30.237 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:30.237 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:55:30.237 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:55:30.237 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:55:30.237 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:30.237 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 144 k 2026-02-25T14:55:30.237 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 802 k 2026-02-25T14:55:30.237 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:55:30.259 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:55:30.260 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:55:30.260 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:55:30.260 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:55:30.260 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:55:30.260 INFO:teuthology.orchestra.run.trial189.stdout: librados2-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 63 M 2026-02-25T14:55:30.260 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:30.260 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:55:30.260 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:55:30.260 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:55:30.260 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:55:30.260 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 63 M 2026-02-25T14:55:30.260 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 397 M 2026-02-25T14:55:30.260 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:55:30.703 INFO:teuthology.orchestra.run.trial093.stdout:librados-devel-debuginfo-20.2.0-676.gc824a7fa.e 309 kB/s | 144 kB 00:00 2026-02-25T14:55:30.704 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:55:30.704 INFO:teuthology.orchestra.run.trial093.stdout:Total 308 kB/s | 144 kB 00:00 2026-02-25T14:55:30.704 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:55:30.705 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:55:30.706 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:55:30.725 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:55:30.725 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:55:30.767 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:55:30.785 INFO:teuthology.orchestra.run.trial093.stdout: Installing : librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:55:30.845 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:55:30.908 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:55:30.908 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:30.908 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:55:30.908 INFO:teuthology.orchestra.run.trial093.stdout: librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:55:30.909 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:30.909 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:55:30.936 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install libcephfs2-debuginfo 2026-02-25T14:55:31.252 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:34:57 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:55:31.335 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:55:31.335 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:55:31.335 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:55:31.335 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:55:31.335 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:55:31.335 INFO:teuthology.orchestra.run.trial093.stdout: libcephfs2-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 9.1 M 2026-02-25T14:55:31.335 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:31.335 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:55:31.335 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:55:31.335 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:55:31.335 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:31.335 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 9.1 M 2026-02-25T14:55:31.335 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 46 M 2026-02-25T14:55:31.335 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:55:35.332 INFO:teuthology.orchestra.run.trial093.stdout:libcephfs2-debuginfo-20.2.0-676.gc824a7fa.el9.x 2.3 MB/s | 9.1 MB 00:03 2026-02-25T14:55:35.332 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:55:35.333 INFO:teuthology.orchestra.run.trial093.stdout:Total 2.3 MB/s | 9.1 MB 00:03 2026-02-25T14:55:35.333 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:55:35.335 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:55:35.335 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:55:35.379 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:55:35.379 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:55:35.879 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:55:35.897 INFO:teuthology.orchestra.run.trial093.stdout: Installing : libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:55:35.992 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:55:36.055 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:55:36.056 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:36.056 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:55:36.056 INFO:teuthology.orchestra.run.trial093.stdout: libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:55:36.056 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:36.056 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:55:36.082 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install librados2-debuginfo 2026-02-25T14:55:36.400 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:35:02 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:55:36.484 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:55:36.484 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:55:36.484 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:55:36.484 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:55:36.484 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:55:36.484 INFO:teuthology.orchestra.run.trial093.stdout: librados2-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 63 M 2026-02-25T14:55:36.485 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:36.485 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:55:36.485 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:55:36.485 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:55:36.485 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:36.485 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 63 M 2026-02-25T14:55:36.485 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 397 M 2026-02-25T14:55:36.485 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:55:52.323 INFO:teuthology.orchestra.run.trial093.stdout:librados2-debuginfo-20.2.0-676.gc824a7fa.el9.x8 4.0 MB/s | 63 MB 00:15 2026-02-25T14:55:52.324 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:55:52.324 INFO:teuthology.orchestra.run.trial093.stdout:Total 4.0 MB/s | 63 MB 00:15 2026-02-25T14:55:52.324 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:55:52.326 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:55:52.326 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:55:52.532 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:55:52.532 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:55:56.034 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:55:56.052 INFO:teuthology.orchestra.run.trial093.stdout: Installing : librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:55:56.397 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:55:56.460 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:55:56.460 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:56.460 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:55:56.460 INFO:teuthology.orchestra.run.trial093.stdout: librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:55:56.460 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:56.461 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:55:56.491 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install librbd1-debuginfo 2026-02-25T14:55:56.805 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:35:22 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:55:56.889 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:55:56.890 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:55:56.890 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:55:56.890 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:55:56.890 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:55:56.890 INFO:teuthology.orchestra.run.trial093.stdout: librbd1-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 80 M 2026-02-25T14:55:56.890 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:56.890 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:55:56.890 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:55:56.890 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:55:56.890 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:55:56.891 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 80 M 2026-02-25T14:55:56.891 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 406 M 2026-02-25T14:55:56.891 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:55:58.286 INFO:teuthology.orchestra.run.trial189.stdout:librados2-debuginfo-20.2.0-676.gc824a7fa.el9.x8 2.3 MB/s | 63 MB 00:28 2026-02-25T14:55:58.287 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:55:58.287 INFO:teuthology.orchestra.run.trial189.stdout:Total 2.3 MB/s | 63 MB 00:28 2026-02-25T14:55:58.287 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:55:58.289 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:55:58.289 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:55:58.496 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:55:58.496 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:56:00.339 INFO:teuthology.orchestra.run.trial094.stdout:librbd1-debuginfo-20.2.0-676.gc824a7fa.el9.x86_ 1.3 MB/s | 80 MB 01:02 2026-02-25T14:56:00.340 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:00.340 INFO:teuthology.orchestra.run.trial094.stdout:Total 1.3 MB/s | 80 MB 01:02 2026-02-25T14:56:00.340 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:56:00.342 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:56:00.342 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:56:00.599 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:56:00.599 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:56:01.992 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:56:02.011 INFO:teuthology.orchestra.run.trial189.stdout: Installing : librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:56:02.360 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:56:02.426 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:56:02.426 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:02.427 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:56:02.427 INFO:teuthology.orchestra.run.trial189.stdout: librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:02.427 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:02.427 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:56:02.458 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install librbd1-debuginfo 2026-02-25T14:56:02.773 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:35:28 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:02.857 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:56:02.858 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:02.858 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:56:02.858 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:02.858 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:56:02.858 INFO:teuthology.orchestra.run.trial189.stdout: librbd1-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 80 M 2026-02-25T14:56:02.858 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:02.858 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:56:02.858 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:02.858 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:56:02.858 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:02.858 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 80 M 2026-02-25T14:56:02.858 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 406 M 2026-02-25T14:56:02.858 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:56:04.762 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:56:04.781 INFO:teuthology.orchestra.run.trial094.stdout: Installing : librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:05.129 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:05.192 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:05.192 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:05.192 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:56:05.192 INFO:teuthology.orchestra.run.trial094.stdout: librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:05.192 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:05.192 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:56:05.223 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-cephfs-debuginfo 2026-02-25T14:56:05.536 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:35:31 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:05.618 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:56:05.618 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:05.618 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:56:05.618 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:05.618 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:56:05.618 INFO:teuthology.orchestra.run.trial094.stdout: python3-cephfs-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 646 k 2026-02-25T14:56:05.619 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:05.619 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:56:05.619 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:05.619 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:56:05.619 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:05.619 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 646 k 2026-02-25T14:56:05.619 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 2.3 M 2026-02-25T14:56:05.619 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:56:06.154 INFO:teuthology.orchestra.run.trial094.stdout:python3-cephfs-debuginfo-20.2.0-676.gc824a7fa.e 1.2 MB/s | 646 kB 00:00 2026-02-25T14:56:06.154 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:06.154 INFO:teuthology.orchestra.run.trial094.stdout:Total 1.2 MB/s | 646 kB 00:00 2026-02-25T14:56:06.154 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:56:06.156 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:56:06.157 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:56:06.176 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:56:06.176 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:56:06.243 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:56:06.261 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:56:06.322 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:56:06.386 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:56:06.386 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:06.386 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:56:06.386 INFO:teuthology.orchestra.run.trial094.stdout: python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:06.386 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:06.386 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:56:06.411 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-rados-debuginfo 2026-02-25T14:56:06.728 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:35:32 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:06.810 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:56:06.811 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:06.811 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:56:06.811 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:06.811 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:56:06.812 INFO:teuthology.orchestra.run.trial094.stdout: python3-rados-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 800 k 2026-02-25T14:56:06.812 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:06.812 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:56:06.812 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:06.812 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:56:06.812 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:06.812 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 800 k 2026-02-25T14:56:06.812 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 3.1 M 2026-02-25T14:56:06.812 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:56:07.295 INFO:teuthology.orchestra.run.trial094.stdout:python3-rados-debuginfo-20.2.0-676.gc824a7fa.el 1.6 MB/s | 800 kB 00:00 2026-02-25T14:56:07.295 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:07.295 INFO:teuthology.orchestra.run.trial094.stdout:Total 1.6 MB/s | 800 kB 00:00 2026-02-25T14:56:07.296 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:56:07.297 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:56:07.297 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:56:07.307 INFO:teuthology.orchestra.run.trial189.stdout:librbd1-debuginfo-20.2.0-676.gc824a7fa.el9.x86_ 18 MB/s | 80 MB 00:04 2026-02-25T14:56:07.307 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:07.308 INFO:teuthology.orchestra.run.trial189.stdout:Total 18 MB/s | 80 MB 00:04 2026-02-25T14:56:07.308 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:56:07.310 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:56:07.310 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:56:07.318 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:56:07.318 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:56:07.393 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:56:07.411 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x 1/1 2026-02-25T14:56:07.473 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x 1/1 2026-02-25T14:56:07.537 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x 1/1 2026-02-25T14:56:07.537 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:07.537 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:56:07.537 INFO:teuthology.orchestra.run.trial094.stdout: python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:07.537 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:07.537 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:56:07.562 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-rbd-debuginfo 2026-02-25T14:56:07.564 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:56:07.564 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:56:07.877 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:35:33 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:07.959 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:56:07.960 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:07.960 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:56:07.960 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:07.960 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:56:07.960 INFO:teuthology.orchestra.run.trial094.stdout: python3-rbd-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 1.2 M 2026-02-25T14:56:07.960 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:07.960 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:56:07.960 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:07.960 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:56:07.960 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:07.960 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 1.2 M 2026-02-25T14:56:07.960 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 4.7 M 2026-02-25T14:56:07.960 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:56:08.348 INFO:teuthology.orchestra.run.trial094.stdout:python3-rbd-debuginfo-20.2.0-676.gc824a7fa.el9. 3.0 MB/s | 1.2 MB 00:00 2026-02-25T14:56:08.349 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:08.349 INFO:teuthology.orchestra.run.trial094.stdout:Total 3.0 MB/s | 1.2 MB 00:00 2026-02-25T14:56:08.349 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:56:08.351 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:56:08.351 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:56:08.372 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:56:08.372 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:56:08.464 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:56:08.482 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:56:08.545 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:56:08.608 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:56:08.608 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:08.608 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:56:08.609 INFO:teuthology.orchestra.run.trial094.stdout: python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:08.609 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:08.609 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:56:08.634 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-rgw-debuginfo 2026-02-25T14:56:08.953 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:35:34 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:09.037 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:56:09.037 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:09.037 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:56:09.038 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:09.038 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:56:09.038 INFO:teuthology.orchestra.run.trial094.stdout: python3-rgw-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 350 k 2026-02-25T14:56:09.038 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:09.038 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:56:09.038 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:09.038 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:56:09.038 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:09.038 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 350 k 2026-02-25T14:56:09.038 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 1.1 M 2026-02-25T14:56:09.038 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:56:09.363 INFO:teuthology.orchestra.run.trial094.stdout:python3-rgw-debuginfo-20.2.0-676.gc824a7fa.el9. 1.1 MB/s | 350 kB 00:00 2026-02-25T14:56:09.363 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:09.363 INFO:teuthology.orchestra.run.trial094.stdout:Total 1.1 MB/s | 350 kB 00:00 2026-02-25T14:56:09.363 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:56:09.365 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:56:09.365 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:56:09.384 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:56:09.384 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:56:09.435 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:56:09.453 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:56:09.512 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:56:09.575 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:56:09.576 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:09.576 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:56:09.576 INFO:teuthology.orchestra.run.trial094.stdout: python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:09.576 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:09.576 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:56:09.602 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install rbd-fuse-debuginfo 2026-02-25T14:56:09.916 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:35:35 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:10.000 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:56:10.001 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:10.001 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:56:10.001 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:10.001 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:56:10.002 INFO:teuthology.orchestra.run.trial094.stdout: rbd-fuse-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 1.5 M 2026-02-25T14:56:10.002 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:10.002 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:56:10.002 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:10.002 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:56:10.002 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:10.002 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 1.5 M 2026-02-25T14:56:10.002 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 7.9 M 2026-02-25T14:56:10.002 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:56:10.260 INFO:teuthology.orchestra.run.trial094.stdout:rbd-fuse-debuginfo-20.2.0-676.gc824a7fa.el9.x86 5.9 MB/s | 1.5 MB 00:00 2026-02-25T14:56:10.261 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:10.261 INFO:teuthology.orchestra.run.trial094.stdout:Total 5.9 MB/s | 1.5 MB 00:00 2026-02-25T14:56:10.261 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:56:10.262 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:56:10.263 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:56:10.285 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:56:10.285 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:56:10.398 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:56:10.416 INFO:teuthology.orchestra.run.trial094.stdout: Installing : rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:10.481 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:10.544 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:10.544 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:10.544 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:56:10.544 INFO:teuthology.orchestra.run.trial094.stdout: rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:10.545 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:10.545 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:56:10.571 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install rbd-mirror-debuginfo 2026-02-25T14:56:10.888 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:35:36 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:10.971 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:56:10.971 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:10.972 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:56:10.972 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:10.972 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:56:10.972 INFO:teuthology.orchestra.run.trial094.stdout: rbd-mirror-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 79 M 2026-02-25T14:56:10.972 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:10.972 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:56:10.972 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:10.972 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:56:10.972 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:10.972 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 79 M 2026-02-25T14:56:10.973 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 375 M 2026-02-25T14:56:10.973 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:56:11.738 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:56:11.757 INFO:teuthology.orchestra.run.trial189.stdout: Installing : librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:12.109 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:12.173 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:12.173 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:12.173 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:56:12.173 INFO:teuthology.orchestra.run.trial189.stdout: librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:12.173 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:12.173 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:56:12.206 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-cephfs-debuginfo 2026-02-25T14:56:12.526 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:35:38 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:12.611 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:56:12.611 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:12.611 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:56:12.611 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:12.611 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:56:12.612 INFO:teuthology.orchestra.run.trial189.stdout: python3-cephfs-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 646 k 2026-02-25T14:56:12.612 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:12.612 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:56:12.612 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:12.612 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:56:12.612 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:12.612 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 646 k 2026-02-25T14:56:12.612 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 2.3 M 2026-02-25T14:56:12.612 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:56:12.991 INFO:teuthology.orchestra.run.trial189.stdout:python3-cephfs-debuginfo-20.2.0-676.gc824a7fa.e 1.7 MB/s | 646 kB 00:00 2026-02-25T14:56:12.991 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:12.991 INFO:teuthology.orchestra.run.trial189.stdout:Total 1.7 MB/s | 646 kB 00:00 2026-02-25T14:56:12.991 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:56:12.993 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:56:12.993 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:56:13.013 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:56:13.013 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:56:13.022 INFO:teuthology.orchestra.run.trial094.stdout:rbd-mirror-debuginfo-20.2.0-676.gc824a7fa.el9.x 39 MB/s | 79 MB 00:02 2026-02-25T14:56:13.023 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:13.023 INFO:teuthology.orchestra.run.trial094.stdout:Total 39 MB/s | 79 MB 00:02 2026-02-25T14:56:13.023 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:56:13.025 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:56:13.025 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:56:13.078 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:56:13.096 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:56:13.158 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:56:13.221 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:56:13.221 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:13.221 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:56:13.221 INFO:teuthology.orchestra.run.trial189.stdout: python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:13.221 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:13.221 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:56:13.247 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-rados-debuginfo 2026-02-25T14:56:13.276 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:56:13.276 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:56:13.565 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:35:39 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:13.647 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:56:13.647 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:13.647 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:56:13.647 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:13.647 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:56:13.647 INFO:teuthology.orchestra.run.trial189.stdout: python3-rados-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 800 k 2026-02-25T14:56:13.647 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:13.647 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:56:13.647 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:13.647 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:56:13.647 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:13.648 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 800 k 2026-02-25T14:56:13.648 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 3.1 M 2026-02-25T14:56:13.648 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:56:14.633 INFO:teuthology.orchestra.run.trial189.stdout:python3-rados-debuginfo-20.2.0-676.gc824a7fa.el 812 kB/s | 800 kB 00:00 2026-02-25T14:56:14.633 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:14.633 INFO:teuthology.orchestra.run.trial189.stdout:Total 811 kB/s | 800 kB 00:00 2026-02-25T14:56:14.633 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:56:14.635 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:56:14.636 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:56:14.656 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:56:14.656 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:56:14.730 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:56:14.749 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x 1/1 2026-02-25T14:56:14.812 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x 1/1 2026-02-25T14:56:14.877 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x 1/1 2026-02-25T14:56:14.877 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:14.877 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:56:14.877 INFO:teuthology.orchestra.run.trial189.stdout: python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:14.877 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:14.877 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:56:14.903 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-rbd-debuginfo 2026-02-25T14:56:15.217 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:35:41 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:15.300 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:56:15.301 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:15.301 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:56:15.301 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:15.301 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:56:15.301 INFO:teuthology.orchestra.run.trial189.stdout: python3-rbd-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 1.2 M 2026-02-25T14:56:15.301 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:15.301 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:56:15.301 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:15.301 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:56:15.301 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:15.301 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 1.2 M 2026-02-25T14:56:15.301 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 4.7 M 2026-02-25T14:56:15.301 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:56:16.154 INFO:teuthology.orchestra.run.trial189.stdout:python3-rbd-debuginfo-20.2.0-676.gc824a7fa.el9. 1.4 MB/s | 1.2 MB 00:00 2026-02-25T14:56:16.155 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:16.155 INFO:teuthology.orchestra.run.trial189.stdout:Total 1.4 MB/s | 1.2 MB 00:00 2026-02-25T14:56:16.155 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:56:16.157 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:56:16.157 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:56:16.178 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:56:16.179 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:56:16.271 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:56:16.289 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:56:16.352 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:56:16.418 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:56:16.418 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:16.418 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:56:16.418 INFO:teuthology.orchestra.run.trial189.stdout: python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:16.418 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:16.418 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:56:16.444 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-rgw-debuginfo 2026-02-25T14:56:16.759 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:35:42 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:16.843 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:56:16.843 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:16.843 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:56:16.843 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:16.843 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:56:16.843 INFO:teuthology.orchestra.run.trial189.stdout: python3-rgw-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 350 k 2026-02-25T14:56:16.843 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:16.843 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:56:16.843 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:16.843 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:56:16.843 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:16.844 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 350 k 2026-02-25T14:56:16.844 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 1.1 M 2026-02-25T14:56:16.844 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:56:17.373 INFO:teuthology.orchestra.run.trial189.stdout:python3-rgw-debuginfo-20.2.0-676.gc824a7fa.el9. 662 kB/s | 350 kB 00:00 2026-02-25T14:56:17.373 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:17.374 INFO:teuthology.orchestra.run.trial189.stdout:Total 661 kB/s | 350 kB 00:00 2026-02-25T14:56:17.374 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:56:17.376 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:56:17.376 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:56:17.395 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:56:17.395 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:56:17.414 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:56:17.433 INFO:teuthology.orchestra.run.trial094.stdout: Installing : rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:56:17.446 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:56:17.464 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:56:17.525 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:56:17.589 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:56:17.589 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:17.589 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:56:17.589 INFO:teuthology.orchestra.run.trial189.stdout: python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:17.589 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:17.589 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:56:17.615 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install rbd-fuse-debuginfo 2026-02-25T14:56:17.768 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:56:17.832 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:56:17.832 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:17.832 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:56:17.832 INFO:teuthology.orchestra.run.trial094.stdout: rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:17.832 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:17.832 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:56:17.865 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install rbd-nbd-debuginfo 2026-02-25T14:56:17.933 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:35:43 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:18.017 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:56:18.018 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:18.018 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:56:18.018 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:18.018 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:56:18.018 INFO:teuthology.orchestra.run.trial189.stdout: rbd-fuse-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 1.5 M 2026-02-25T14:56:18.018 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:18.018 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:56:18.018 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:18.018 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:56:18.018 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:18.018 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 1.5 M 2026-02-25T14:56:18.018 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 7.9 M 2026-02-25T14:56:18.018 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:56:18.177 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:35:44 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:18.260 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T14:56:18.261 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:18.261 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T14:56:18.261 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:18.261 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T14:56:18.261 INFO:teuthology.orchestra.run.trial094.stdout: rbd-nbd-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.4 M 2026-02-25T14:56:18.261 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:18.261 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T14:56:18.261 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T14:56:18.261 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T14:56:18.261 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:18.261 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 2.4 M 2026-02-25T14:56:18.261 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 12 M 2026-02-25T14:56:18.261 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T14:56:18.797 INFO:teuthology.orchestra.run.trial189.stdout:rbd-fuse-debuginfo-20.2.0-676.gc824a7fa.el9.x86 2.0 MB/s | 1.5 MB 00:00 2026-02-25T14:56:18.797 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:18.797 INFO:teuthology.orchestra.run.trial189.stdout:Total 2.0 MB/s | 1.5 MB 00:00 2026-02-25T14:56:18.797 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:56:18.799 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:56:18.799 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:56:18.821 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:56:18.821 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:56:18.935 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:56:18.953 INFO:teuthology.orchestra.run.trial189.stdout: Installing : rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:19.021 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:19.086 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:19.086 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:19.086 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:56:19.086 INFO:teuthology.orchestra.run.trial189.stdout: rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:19.086 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:19.086 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:56:19.115 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install rbd-mirror-debuginfo 2026-02-25T14:56:19.432 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:35:45 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:19.514 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:56:19.515 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:19.516 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:56:19.516 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:19.516 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:56:19.516 INFO:teuthology.orchestra.run.trial189.stdout: rbd-mirror-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 79 M 2026-02-25T14:56:19.516 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:19.516 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:56:19.516 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:19.516 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:56:19.516 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:19.516 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 79 M 2026-02-25T14:56:19.517 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 375 M 2026-02-25T14:56:19.517 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:56:21.669 INFO:teuthology.orchestra.run.trial094.stdout:rbd-nbd-debuginfo-20.2.0-676.gc824a7fa.el9.x86_ 733 kB/s | 2.4 MB 00:03 2026-02-25T14:56:21.671 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:21.671 INFO:teuthology.orchestra.run.trial094.stdout:Total 732 kB/s | 2.4 MB 00:03 2026-02-25T14:56:21.671 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T14:56:21.673 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T14:56:21.673 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T14:56:21.698 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T14:56:21.698 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T14:56:21.860 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T14:56:21.878 INFO:teuthology.orchestra.run.trial094.stdout: Installing : rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:21.951 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:22.015 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:22.015 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:22.015 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T14:56:22.015 INFO:teuthology.orchestra.run.trial094.stdout: rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:22.015 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T14:56:22.015 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T14:56:22.048 DEBUG:teuthology.parallel:result is None 2026-02-25T14:56:33.343 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:56:33.363 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:56:35.750 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:56:35.814 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:56:35.814 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:56:35.814 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:56:35.814 INFO:teuthology.orchestra.run.trial090.stdout: ceph-test-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:35.814 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:56:35.815 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:56:36.020 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-base-debuginfo 2026-02-25T14:56:36.339 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:36:02 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:36.404 INFO:teuthology.orchestra.run.trial090.stdout:Package ceph-base-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 is already installed. 2026-02-25T14:56:36.422 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:56:36.423 INFO:teuthology.orchestra.run.trial090.stdout:Nothing to do. 2026-02-25T14:56:36.423 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:56:36.445 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-mgr-debuginfo 2026-02-25T14:56:36.581 INFO:teuthology.orchestra.run.trial189.stdout:rbd-mirror-debuginfo-20.2.0-676.gc824a7fa.el9.x 4.6 MB/s | 79 MB 00:17 2026-02-25T14:56:36.581 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:36.582 INFO:teuthology.orchestra.run.trial189.stdout:Total 4.6 MB/s | 79 MB 00:17 2026-02-25T14:56:36.582 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:56:36.583 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:56:36.584 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:56:36.764 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:36:02 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:36.834 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:56:36.835 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:56:36.848 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:56:36.848 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:56:36.849 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:56:36.849 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:56:36.849 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:56:36.849 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 22 M 2026-02-25T14:56:36.849 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:56:36.849 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:56:36.849 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:56:36.849 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:56:36.849 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:56:36.849 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 22 M 2026-02-25T14:56:36.849 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 146 M 2026-02-25T14:56:36.850 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:56:39.822 INFO:teuthology.orchestra.run.trial090.stdout:ceph-mgr-debuginfo-20.2.0-676.gc824a7fa.el9.x86 7.4 MB/s | 22 MB 00:02 2026-02-25T14:56:39.823 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:39.823 INFO:teuthology.orchestra.run.trial090.stdout:Total 7.4 MB/s | 22 MB 00:02 2026-02-25T14:56:39.823 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:56:39.824 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:56:39.825 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:56:39.909 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:56:39.910 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:56:40.878 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:56:40.896 INFO:teuthology.orchestra.run.trial189.stdout: Installing : rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:56:41.169 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:56:41.188 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:41.234 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:56:41.299 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:56:41.299 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:41.300 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:56:41.300 INFO:teuthology.orchestra.run.trial189.stdout: rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:41.300 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:41.300 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:56:41.335 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install rbd-nbd-debuginfo 2026-02-25T14:56:41.356 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:41.419 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:41.419 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:56:41.419 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:56:41.419 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mgr-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:41.419 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:56:41.419 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:56:41.448 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-mds-debuginfo 2026-02-25T14:56:41.656 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:36:07 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:41.741 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-25T14:56:41.741 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:41.741 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-25T14:56:41.741 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:41.741 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-25T14:56:41.741 INFO:teuthology.orchestra.run.trial189.stdout: rbd-nbd-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.4 M 2026-02-25T14:56:41.742 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:41.742 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-25T14:56:41.742 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-25T14:56:41.742 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-25T14:56:41.742 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:41.742 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 2.4 M 2026-02-25T14:56:41.742 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 12 M 2026-02-25T14:56:41.742 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-25T14:56:41.769 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:36:07 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:41.852 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:56:41.853 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:56:41.853 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:56:41.853 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:56:41.853 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:56:41.853 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mds-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 54 M 2026-02-25T14:56:41.854 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:56:41.854 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:56:41.854 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:56:41.854 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:56:41.854 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:56:41.854 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 54 M 2026-02-25T14:56:41.854 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 221 M 2026-02-25T14:56:41.854 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:56:42.771 INFO:teuthology.orchestra.run.trial189.stdout:rbd-nbd-debuginfo-20.2.0-676.gc824a7fa.el9.x86_ 2.4 MB/s | 2.4 MB 00:01 2026-02-25T14:56:42.771 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:42.771 INFO:teuthology.orchestra.run.trial189.stdout:Total 2.4 MB/s | 2.4 MB 00:01 2026-02-25T14:56:42.772 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-25T14:56:42.773 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-25T14:56:42.773 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-25T14:56:42.798 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-25T14:56:42.798 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-25T14:56:42.960 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-25T14:56:42.978 INFO:teuthology.orchestra.run.trial189.stdout: Installing : rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:43.053 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:43.117 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:43.117 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:43.118 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-25T14:56:43.118 INFO:teuthology.orchestra.run.trial189.stdout: rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:43.118 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T14:56:43.118 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-25T14:56:43.149 DEBUG:teuthology.parallel:result is None 2026-02-25T14:56:50.389 INFO:teuthology.orchestra.run.trial093.stdout:librbd1-debuginfo-20.2.0-676.gc824a7fa.el9.x86_ 1.5 MB/s | 80 MB 00:53 2026-02-25T14:56:50.390 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:50.391 INFO:teuthology.orchestra.run.trial093.stdout:Total 1.5 MB/s | 80 MB 00:53 2026-02-25T14:56:50.391 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:56:50.392 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:56:50.392 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:56:50.650 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:56:50.651 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:56:55.040 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:56:55.059 INFO:teuthology.orchestra.run.trial093.stdout: Installing : librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:55.417 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:55.482 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:56:55.482 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:56:55.482 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:56:55.482 INFO:teuthology.orchestra.run.trial093.stdout: librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:55.482 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:56:55.482 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:56:55.517 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install python3-cephfs-debuginfo 2026-02-25T14:56:55.844 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:36:21 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:55.927 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:56:55.928 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:56:55.928 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:56:55.928 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:56:55.929 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:56:55.929 INFO:teuthology.orchestra.run.trial093.stdout: python3-cephfs-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 646 k 2026-02-25T14:56:55.929 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:56:55.929 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:56:55.929 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:56:55.929 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:56:55.929 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:56:55.929 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 646 k 2026-02-25T14:56:55.929 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 2.3 M 2026-02-25T14:56:55.929 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:56:56.478 INFO:teuthology.orchestra.run.trial093.stdout:python3-cephfs-debuginfo-20.2.0-676.gc824a7fa.e 1.1 MB/s | 646 kB 00:00 2026-02-25T14:56:56.479 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:56.479 INFO:teuthology.orchestra.run.trial093.stdout:Total 1.1 MB/s | 646 kB 00:00 2026-02-25T14:56:56.479 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:56:56.481 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:56:56.481 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:56:56.501 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:56:56.501 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:56:56.567 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:56:56.585 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:56:56.647 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:56:56.711 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:56:56.711 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:56:56.711 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:56:56.711 INFO:teuthology.orchestra.run.trial093.stdout: python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:56.711 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:56:56.711 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:56:56.738 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install python3-rados-debuginfo 2026-02-25T14:56:57.057 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:36:23 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:57.140 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:56:57.141 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:56:57.141 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:56:57.142 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:56:57.142 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:56:57.142 INFO:teuthology.orchestra.run.trial093.stdout: python3-rados-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 800 k 2026-02-25T14:56:57.142 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:56:57.142 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:56:57.142 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:56:57.142 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:56:57.142 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:56:57.142 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 800 k 2026-02-25T14:56:57.142 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 3.1 M 2026-02-25T14:56:57.143 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:56:58.274 INFO:teuthology.orchestra.run.trial093.stdout:python3-rados-debuginfo-20.2.0-676.gc824a7fa.el 706 kB/s | 800 kB 00:01 2026-02-25T14:56:58.275 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:56:58.275 INFO:teuthology.orchestra.run.trial093.stdout:Total 706 kB/s | 800 kB 00:01 2026-02-25T14:56:58.275 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:56:58.277 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:56:58.277 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:56:58.297 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:56:58.297 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:56:58.371 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:56:58.388 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x 1/1 2026-02-25T14:56:58.450 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x 1/1 2026-02-25T14:56:58.513 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x 1/1 2026-02-25T14:56:58.513 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:56:58.513 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:56:58.513 INFO:teuthology.orchestra.run.trial093.stdout: python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:56:58.514 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:56:58.514 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:56:58.539 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install python3-rbd-debuginfo 2026-02-25T14:56:58.851 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:36:24 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:56:58.934 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:56:58.935 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:56:58.935 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:56:58.935 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:56:58.935 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:56:58.935 INFO:teuthology.orchestra.run.trial093.stdout: python3-rbd-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 1.2 M 2026-02-25T14:56:58.935 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:56:58.935 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:56:58.936 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:56:58.936 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:56:58.936 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:56:58.936 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 1.2 M 2026-02-25T14:56:58.936 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 4.7 M 2026-02-25T14:56:58.936 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:57:01.053 INFO:teuthology.orchestra.run.trial093.stdout:python3-rbd-debuginfo-20.2.0-676.gc824a7fa.el9. 563 kB/s | 1.2 MB 00:02 2026-02-25T14:57:01.054 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:57:01.054 INFO:teuthology.orchestra.run.trial093.stdout:Total 563 kB/s | 1.2 MB 00:02 2026-02-25T14:57:01.054 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:57:01.056 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:57:01.056 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:57:01.077 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:57:01.077 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:57:01.169 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:57:01.187 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:57:01.249 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:57:01.313 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:57:01.313 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:01.314 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:57:01.314 INFO:teuthology.orchestra.run.trial093.stdout: python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:57:01.314 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:01.314 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:57:01.340 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install python3-rgw-debuginfo 2026-02-25T14:57:01.652 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:36:27 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:57:01.735 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:57:01.736 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:57:01.736 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:57:01.737 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:57:01.737 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:57:01.737 INFO:teuthology.orchestra.run.trial093.stdout: python3-rgw-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 350 k 2026-02-25T14:57:01.737 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:01.737 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:57:01.737 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:57:01.737 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:57:01.737 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:01.737 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 350 k 2026-02-25T14:57:01.737 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 1.1 M 2026-02-25T14:57:01.738 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:57:02.493 INFO:teuthology.orchestra.run.trial093.stdout:python3-rgw-debuginfo-20.2.0-676.gc824a7fa.el9. 463 kB/s | 350 kB 00:00 2026-02-25T14:57:02.493 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:57:02.493 INFO:teuthology.orchestra.run.trial093.stdout:Total 463 kB/s | 350 kB 00:00 2026-02-25T14:57:02.493 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:57:02.495 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:57:02.495 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:57:02.514 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:57:02.514 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:57:02.566 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:57:02.584 INFO:teuthology.orchestra.run.trial093.stdout: Installing : python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:57:02.645 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:57:02.709 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:57:02.709 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:02.709 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:57:02.709 INFO:teuthology.orchestra.run.trial093.stdout: python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:57:02.710 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:02.710 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:57:02.736 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install rbd-fuse-debuginfo 2026-02-25T14:57:03.053 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:36:29 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:57:03.137 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:57:03.137 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:57:03.137 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:57:03.137 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:57:03.137 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:57:03.137 INFO:teuthology.orchestra.run.trial093.stdout: rbd-fuse-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 1.5 M 2026-02-25T14:57:03.138 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:03.138 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:57:03.138 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:57:03.138 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:57:03.138 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:03.138 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 1.5 M 2026-02-25T14:57:03.138 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 7.9 M 2026-02-25T14:57:03.138 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:57:03.963 INFO:teuthology.orchestra.run.trial093.stdout:rbd-fuse-debuginfo-20.2.0-676.gc824a7fa.el9.x86 1.9 MB/s | 1.5 MB 00:00 2026-02-25T14:57:03.963 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:57:03.963 INFO:teuthology.orchestra.run.trial093.stdout:Total 1.9 MB/s | 1.5 MB 00:00 2026-02-25T14:57:03.963 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:57:03.966 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:57:03.966 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:57:03.988 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:57:03.988 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:57:04.102 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:57:04.119 INFO:teuthology.orchestra.run.trial093.stdout: Installing : rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:57:04.186 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:57:04.249 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:57:04.249 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:04.249 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:57:04.249 INFO:teuthology.orchestra.run.trial093.stdout: rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:57:04.249 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:04.250 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:57:04.275 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install rbd-mirror-debuginfo 2026-02-25T14:57:04.595 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:36:30 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:57:04.678 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:57:04.679 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:57:04.679 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:57:04.679 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:57:04.679 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:57:04.679 INFO:teuthology.orchestra.run.trial093.stdout: rbd-mirror-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 79 M 2026-02-25T14:57:04.679 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:04.679 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:57:04.679 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:57:04.679 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:57:04.679 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:04.680 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 79 M 2026-02-25T14:57:04.680 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 375 M 2026-02-25T14:57:04.680 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:57:08.077 INFO:teuthology.orchestra.run.trial090.stdout:ceph-mds-debuginfo-20.2.0-676.gc824a7fa.el9.x86 2.1 MB/s | 54 MB 00:26 2026-02-25T14:57:08.078 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:57:08.078 INFO:teuthology.orchestra.run.trial090.stdout:Total 2.1 MB/s | 54 MB 00:26 2026-02-25T14:57:08.078 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:57:08.080 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:57:08.080 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:57:08.256 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:57:08.257 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:57:10.903 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:57:10.921 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:57:11.141 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:57:11.204 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:57:11.205 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:57:11.205 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:57:11.205 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mds-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:57:11.205 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:57:11.205 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:57:11.236 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-mon-debuginfo 2026-02-25T14:57:11.555 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:36:37 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:57:11.638 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:57:11.639 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:57:11.639 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:57:11.639 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:57:11.639 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:57:11.639 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mon-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 129 M 2026-02-25T14:57:11.640 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:57:11.640 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:57:11.640 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:57:11.640 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:57:11.640 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:57:11.640 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 129 M 2026-02-25T14:57:11.640 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 641 M 2026-02-25T14:57:11.640 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:57:34.582 INFO:teuthology.orchestra.run.trial093.stdout:rbd-mirror-debuginfo-20.2.0-676.gc824a7fa.el9.x 2.7 MB/s | 79 MB 00:29 2026-02-25T14:57:34.583 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:57:34.583 INFO:teuthology.orchestra.run.trial093.stdout:Total 2.7 MB/s | 79 MB 00:29 2026-02-25T14:57:34.584 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:57:34.585 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:57:34.585 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:57:34.836 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:57:34.836 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:57:38.874 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:57:38.892 INFO:teuthology.orchestra.run.trial093.stdout: Installing : rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:57:39.223 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:57:39.287 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:57:39.287 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:39.287 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:57:39.287 INFO:teuthology.orchestra.run.trial093.stdout: rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:57:39.287 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:39.288 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:57:39.320 DEBUG:teuthology.orchestra.run.trial093:> sudo yum -y install rbd-nbd-debuginfo 2026-02-25T14:57:39.639 INFO:teuthology.orchestra.run.trial093.stdout:Last metadata expiration check: 0:37:05 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:57:39.722 INFO:teuthology.orchestra.run.trial093.stdout:Dependencies resolved. 2026-02-25T14:57:39.723 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:57:39.724 INFO:teuthology.orchestra.run.trial093.stdout: Package Arch Version Repo Size 2026-02-25T14:57:39.724 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:57:39.724 INFO:teuthology.orchestra.run.trial093.stdout:Installing: 2026-02-25T14:57:39.724 INFO:teuthology.orchestra.run.trial093.stdout: rbd-nbd-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.4 M 2026-02-25T14:57:39.724 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:39.724 INFO:teuthology.orchestra.run.trial093.stdout:Transaction Summary 2026-02-25T14:57:39.724 INFO:teuthology.orchestra.run.trial093.stdout:================================================================================ 2026-02-25T14:57:39.724 INFO:teuthology.orchestra.run.trial093.stdout:Install 1 Package 2026-02-25T14:57:39.724 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:39.724 INFO:teuthology.orchestra.run.trial093.stdout:Total download size: 2.4 M 2026-02-25T14:57:39.725 INFO:teuthology.orchestra.run.trial093.stdout:Installed size: 12 M 2026-02-25T14:57:39.725 INFO:teuthology.orchestra.run.trial093.stdout:Downloading Packages: 2026-02-25T14:57:40.838 INFO:teuthology.orchestra.run.trial093.stdout:rbd-nbd-debuginfo-20.2.0-676.gc824a7fa.el9.x86_ 2.2 MB/s | 2.4 MB 00:01 2026-02-25T14:57:40.838 INFO:teuthology.orchestra.run.trial093.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:57:40.838 INFO:teuthology.orchestra.run.trial093.stdout:Total 2.2 MB/s | 2.4 MB 00:01 2026-02-25T14:57:40.838 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction check 2026-02-25T14:57:40.839 INFO:teuthology.orchestra.run.trial093.stdout:Transaction check succeeded. 2026-02-25T14:57:40.839 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction test 2026-02-25T14:57:40.864 INFO:teuthology.orchestra.run.trial093.stdout:Transaction test succeeded. 2026-02-25T14:57:40.864 INFO:teuthology.orchestra.run.trial093.stdout:Running transaction 2026-02-25T14:57:41.023 INFO:teuthology.orchestra.run.trial093.stdout: Preparing : 1/1 2026-02-25T14:57:41.041 INFO:teuthology.orchestra.run.trial093.stdout: Installing : rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:57:41.111 INFO:teuthology.orchestra.run.trial093.stdout: Running scriptlet: rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:57:41.175 INFO:teuthology.orchestra.run.trial093.stdout: Verifying : rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:57:41.175 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:41.175 INFO:teuthology.orchestra.run.trial093.stdout:Installed: 2026-02-25T14:57:41.175 INFO:teuthology.orchestra.run.trial093.stdout: rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:57:41.175 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T14:57:41.175 INFO:teuthology.orchestra.run.trial093.stdout:Complete! 2026-02-25T14:57:41.206 DEBUG:teuthology.parallel:result is None 2026-02-25T14:57:41.259 INFO:teuthology.orchestra.run.trial090.stdout:ceph-mon-debuginfo-20.2.0-676.gc824a7fa.el9.x86 4.4 MB/s | 129 MB 00:29 2026-02-25T14:57:41.260 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:57:41.260 INFO:teuthology.orchestra.run.trial090.stdout:Total 4.4 MB/s | 129 MB 00:29 2026-02-25T14:57:41.260 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:57:41.262 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:57:41.262 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:57:41.660 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:57:41.661 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:57:48.277 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:57:48.296 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:57:48.832 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:57:48.897 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:57:48.897 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:57:48.897 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:57:48.897 INFO:teuthology.orchestra.run.trial090.stdout: ceph-mon-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:57:48.897 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:57:48.897 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:57:48.935 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-osd-debuginfo 2026-02-25T14:57:49.255 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:37:15 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:57:49.340 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:57:49.340 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:57:49.340 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:57:49.340 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:57:49.340 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:57:49.340 INFO:teuthology.orchestra.run.trial090.stdout: ceph-osd-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 343 M 2026-02-25T14:57:49.340 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:57:49.340 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:57:49.341 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:57:49.341 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:57:49.341 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:57:49.341 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 343 M 2026-02-25T14:57:49.341 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 1.6 G 2026-02-25T14:57:49.341 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:58:44.215 INFO:teuthology.orchestra.run.trial090.stdout:ceph-osd-debuginfo-20.2.0-676.gc824a7fa.el9.x86 6.3 MB/s | 343 MB 00:54 2026-02-25T14:58:44.216 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:58:44.216 INFO:teuthology.orchestra.run.trial090.stdout:Total 6.3 MB/s | 343 MB 00:54 2026-02-25T14:58:44.216 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:58:44.218 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:58:44.218 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:58:45.246 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:58:45.247 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:59:02.501 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:59:02.519 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:59:03.752 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:59:03.817 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:59:03.817 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:03.817 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:59:03.817 INFO:teuthology.orchestra.run.trial090.stdout: ceph-osd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:59:03.817 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:03.817 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:59:03.876 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install ceph-fuse-debuginfo 2026-02-25T14:59:04.206 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:38:30 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:59:04.291 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:59:04.291 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:04.292 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:59:04.292 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:04.292 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:59:04.292 INFO:teuthology.orchestra.run.trial090.stdout: ceph-fuse-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 10 M 2026-02-25T14:59:04.292 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:04.292 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:59:04.292 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:04.292 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:59:04.292 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:04.292 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 10 M 2026-02-25T14:59:04.292 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 51 M 2026-02-25T14:59:04.292 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:59:06.006 INFO:teuthology.orchestra.run.trial090.stdout:ceph-fuse-debuginfo-20.2.0-676.gc824a7fa.el9.x8 5.9 MB/s | 10 MB 00:01 2026-02-25T14:59:06.007 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:59:06.008 INFO:teuthology.orchestra.run.trial090.stdout:Total 5.9 MB/s | 10 MB 00:01 2026-02-25T14:59:06.008 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:59:06.009 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:59:06.009 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:59:06.056 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:59:06.056 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:59:06.610 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:59:06.629 INFO:teuthology.orchestra.run.trial090.stdout: Installing : ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:59:06.729 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:59:06.794 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:59:06.794 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:06.794 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:59:06.794 INFO:teuthology.orchestra.run.trial090.stdout: ceph-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:59:06.795 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:06.795 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:59:06.824 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install librados-devel-debuginfo 2026-02-25T14:59:07.147 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:38:33 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:59:07.232 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:59:07.232 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:07.232 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:59:07.233 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:07.233 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:59:07.233 INFO:teuthology.orchestra.run.trial090.stdout: librados-devel-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 144 k 2026-02-25T14:59:07.233 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:07.233 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:59:07.233 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:07.233 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:59:07.233 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:07.233 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 144 k 2026-02-25T14:59:07.233 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 802 k 2026-02-25T14:59:07.234 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:59:07.685 INFO:teuthology.orchestra.run.trial090.stdout:librados-devel-debuginfo-20.2.0-676.gc824a7fa.e 318 kB/s | 144 kB 00:00 2026-02-25T14:59:07.685 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:59:07.685 INFO:teuthology.orchestra.run.trial090.stdout:Total 317 kB/s | 144 kB 00:00 2026-02-25T14:59:07.685 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:59:07.687 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:59:07.687 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:59:07.707 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:59:07.707 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:59:07.749 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:59:07.767 INFO:teuthology.orchestra.run.trial090.stdout: Installing : librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:59:07.830 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:59:07.893 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:59:07.893 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:07.893 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:59:07.893 INFO:teuthology.orchestra.run.trial090.stdout: librados-devel-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:59:07.893 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:07.893 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:59:07.921 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install libcephfs2-debuginfo 2026-02-25T14:59:08.236 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:38:34 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:59:08.320 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:59:08.320 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:08.320 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:59:08.321 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:08.321 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:59:08.321 INFO:teuthology.orchestra.run.trial090.stdout: libcephfs2-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 9.1 M 2026-02-25T14:59:08.321 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:08.321 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:59:08.321 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:08.321 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:59:08.321 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:08.321 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 9.1 M 2026-02-25T14:59:08.321 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 46 M 2026-02-25T14:59:08.321 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:59:09.922 INFO:teuthology.orchestra.run.trial090.stdout:libcephfs2-debuginfo-20.2.0-676.gc824a7fa.el9.x 5.7 MB/s | 9.1 MB 00:01 2026-02-25T14:59:09.923 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:59:09.923 INFO:teuthology.orchestra.run.trial090.stdout:Total 5.7 MB/s | 9.1 MB 00:01 2026-02-25T14:59:09.923 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:59:09.925 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:59:09.925 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:59:09.970 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:59:09.970 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:59:10.480 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:59:10.498 INFO:teuthology.orchestra.run.trial090.stdout: Installing : libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:59:10.595 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:59:10.660 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T14:59:10.660 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:10.660 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:59:10.660 INFO:teuthology.orchestra.run.trial090.stdout: libcephfs2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:59:10.660 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:10.660 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:59:10.689 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install librados2-debuginfo 2026-02-25T14:59:11.010 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:38:37 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:59:11.094 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:59:11.095 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:11.095 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:59:11.095 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:11.095 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:59:11.095 INFO:teuthology.orchestra.run.trial090.stdout: librados2-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 63 M 2026-02-25T14:59:11.095 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:11.095 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:59:11.095 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:11.095 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:59:11.096 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:11.096 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 63 M 2026-02-25T14:59:11.096 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 397 M 2026-02-25T14:59:11.096 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:59:29.346 INFO:teuthology.orchestra.run.trial090.stdout:librados2-debuginfo-20.2.0-676.gc824a7fa.el9.x8 3.5 MB/s | 63 MB 00:18 2026-02-25T14:59:29.347 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:59:29.347 INFO:teuthology.orchestra.run.trial090.stdout:Total 3.5 MB/s | 63 MB 00:18 2026-02-25T14:59:29.347 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:59:29.349 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:59:29.349 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:59:29.556 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:59:29.556 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:59:33.066 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:59:33.084 INFO:teuthology.orchestra.run.trial090.stdout: Installing : librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:59:33.433 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:59:33.499 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_6 1/1 2026-02-25T14:59:33.499 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:33.499 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:59:33.500 INFO:teuthology.orchestra.run.trial090.stdout: librados2-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:59:33.500 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:33.500 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:59:33.533 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install librbd1-debuginfo 2026-02-25T14:59:33.848 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:38:59 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:59:33.932 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:59:33.932 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:33.932 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:59:33.933 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:33.933 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:59:33.933 INFO:teuthology.orchestra.run.trial090.stdout: librbd1-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 80 M 2026-02-25T14:59:33.933 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:33.933 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:59:33.933 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:33.933 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:59:33.933 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:33.933 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 80 M 2026-02-25T14:59:33.933 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 406 M 2026-02-25T14:59:33.934 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:59:48.553 INFO:teuthology.orchestra.run.trial090.stdout:librbd1-debuginfo-20.2.0-676.gc824a7fa.el9.x86_ 5.5 MB/s | 80 MB 00:14 2026-02-25T14:59:48.554 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:59:48.554 INFO:teuthology.orchestra.run.trial090.stdout:Total 5.5 MB/s | 80 MB 00:14 2026-02-25T14:59:48.554 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:59:48.556 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:59:48.556 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:59:48.813 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:59:48.813 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:59:52.987 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:59:53.006 INFO:teuthology.orchestra.run.trial090.stdout: Installing : librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:59:53.358 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:59:53.422 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:59:53.422 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:53.422 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:59:53.422 INFO:teuthology.orchestra.run.trial090.stdout: librbd1-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:59:53.422 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:53.423 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:59:53.456 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install python3-cephfs-debuginfo 2026-02-25T14:59:53.776 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:39:19 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:59:53.859 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:59:53.860 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:53.860 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:59:53.860 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:53.860 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:59:53.860 INFO:teuthology.orchestra.run.trial090.stdout: python3-cephfs-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 646 k 2026-02-25T14:59:53.860 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:53.860 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:59:53.861 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:53.861 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:59:53.861 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:53.861 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 646 k 2026-02-25T14:59:53.861 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 2.3 M 2026-02-25T14:59:53.861 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:59:54.553 INFO:teuthology.orchestra.run.trial090.stdout:python3-cephfs-debuginfo-20.2.0-676.gc824a7fa.e 932 kB/s | 646 kB 00:00 2026-02-25T14:59:54.554 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:59:54.554 INFO:teuthology.orchestra.run.trial090.stdout:Total 929 kB/s | 646 kB 00:00 2026-02-25T14:59:54.555 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:59:54.557 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:59:54.557 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:59:54.577 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:59:54.578 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:59:54.643 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:59:54.660 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:59:54.722 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:59:54.786 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9. 1/1 2026-02-25T14:59:54.786 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:54.787 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:59:54.787 INFO:teuthology.orchestra.run.trial090.stdout: python3-cephfs-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:59:54.787 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:54.787 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:59:54.812 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install python3-rados-debuginfo 2026-02-25T14:59:55.134 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:39:21 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:59:55.217 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:59:55.218 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:55.218 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:59:55.218 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:55.218 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:59:55.218 INFO:teuthology.orchestra.run.trial090.stdout: python3-rados-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 800 k 2026-02-25T14:59:55.218 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:55.218 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:59:55.218 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:55.218 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:59:55.219 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:55.219 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 800 k 2026-02-25T14:59:55.219 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 3.1 M 2026-02-25T14:59:55.219 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:59:55.726 INFO:teuthology.orchestra.run.trial090.stdout:python3-rados-debuginfo-20.2.0-676.gc824a7fa.el 1.5 MB/s | 800 kB 00:00 2026-02-25T14:59:55.727 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:59:55.727 INFO:teuthology.orchestra.run.trial090.stdout:Total 1.5 MB/s | 800 kB 00:00 2026-02-25T14:59:55.727 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:59:55.728 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:59:55.728 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:59:55.749 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:59:55.750 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:59:55.823 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:59:55.841 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x 1/1 2026-02-25T14:59:55.906 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x 1/1 2026-02-25T14:59:55.970 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x 1/1 2026-02-25T14:59:55.970 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:55.970 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:59:55.970 INFO:teuthology.orchestra.run.trial090.stdout: python3-rados-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:59:55.971 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:55.971 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:59:55.998 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install python3-rbd-debuginfo 2026-02-25T14:59:56.316 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:39:22 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:59:56.399 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:59:56.400 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:56.400 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:59:56.400 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:56.400 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:59:56.400 INFO:teuthology.orchestra.run.trial090.stdout: python3-rbd-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 1.2 M 2026-02-25T14:59:56.400 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:56.400 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:59:56.401 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:56.401 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:59:56.401 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:56.401 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 1.2 M 2026-02-25T14:59:56.401 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 4.7 M 2026-02-25T14:59:56.401 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:59:56.968 INFO:teuthology.orchestra.run.trial090.stdout:python3-rbd-debuginfo-20.2.0-676.gc824a7fa.el9. 2.1 MB/s | 1.2 MB 00:00 2026-02-25T14:59:56.968 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:59:56.968 INFO:teuthology.orchestra.run.trial090.stdout:Total 2.0 MB/s | 1.2 MB 00:00 2026-02-25T14:59:56.968 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:59:56.970 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:59:56.970 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:59:56.991 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:59:56.991 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:59:57.084 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:59:57.101 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:59:57.168 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:59:57.232 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:59:57.232 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:57.232 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:59:57.233 INFO:teuthology.orchestra.run.trial090.stdout: python3-rbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:59:57.233 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:57.233 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:59:57.262 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install python3-rgw-debuginfo 2026-02-25T14:59:57.576 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:39:23 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:59:57.659 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:59:57.660 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:57.660 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:59:57.660 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:57.660 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:59:57.660 INFO:teuthology.orchestra.run.trial090.stdout: python3-rgw-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 350 k 2026-02-25T14:59:57.660 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:57.660 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:59:57.660 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:57.661 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:59:57.661 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:57.661 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 350 k 2026-02-25T14:59:57.661 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 1.1 M 2026-02-25T14:59:57.661 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:59:58.095 INFO:teuthology.orchestra.run.trial090.stdout:python3-rgw-debuginfo-20.2.0-676.gc824a7fa.el9. 804 kB/s | 350 kB 00:00 2026-02-25T14:59:58.096 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:59:58.096 INFO:teuthology.orchestra.run.trial090.stdout:Total 802 kB/s | 350 kB 00:00 2026-02-25T14:59:58.096 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:59:58.098 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:59:58.098 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:59:58.117 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:59:58.117 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:59:58.169 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:59:58.187 INFO:teuthology.orchestra.run.trial090.stdout: Installing : python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:59:58.251 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:59:58.316 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86 1/1 2026-02-25T14:59:58.316 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:58.317 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:59:58.317 INFO:teuthology.orchestra.run.trial090.stdout: python3-rgw-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:59:58.317 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:58.317 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:59:58.346 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install rbd-fuse-debuginfo 2026-02-25T14:59:58.666 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:39:24 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T14:59:58.749 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T14:59:58.750 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:58.750 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T14:59:58.750 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:58.751 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T14:59:58.751 INFO:teuthology.orchestra.run.trial090.stdout: rbd-fuse-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 1.5 M 2026-02-25T14:59:58.751 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:58.751 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T14:59:58.751 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T14:59:58.751 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T14:59:58.751 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:58.751 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 1.5 M 2026-02-25T14:59:58.751 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 7.9 M 2026-02-25T14:59:58.751 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T14:59:59.332 INFO:teuthology.orchestra.run.trial090.stdout:rbd-fuse-debuginfo-20.2.0-676.gc824a7fa.el9.x86 2.6 MB/s | 1.5 MB 00:00 2026-02-25T14:59:59.333 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T14:59:59.333 INFO:teuthology.orchestra.run.trial090.stdout:Total 2.6 MB/s | 1.5 MB 00:00 2026-02-25T14:59:59.333 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T14:59:59.335 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T14:59:59.335 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T14:59:59.357 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T14:59:59.357 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T14:59:59.471 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T14:59:59.489 INFO:teuthology.orchestra.run.trial090.stdout: Installing : rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:59:59.557 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:59:59.621 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T14:59:59.621 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:59.621 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T14:59:59.621 INFO:teuthology.orchestra.run.trial090.stdout: rbd-fuse-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T14:59:59.621 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T14:59:59.621 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T14:59:59.649 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install rbd-mirror-debuginfo 2026-02-25T14:59:59.970 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:39:25 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T15:00:00.053 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T15:00:00.066 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T15:00:00.066 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T15:00:00.066 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T15:00:00.067 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T15:00:00.067 INFO:teuthology.orchestra.run.trial090.stdout: rbd-mirror-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 79 M 2026-02-25T15:00:00.067 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:00.067 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T15:00:00.067 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T15:00:00.067 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T15:00:00.067 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:00.067 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 79 M 2026-02-25T15:00:00.067 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 375 M 2026-02-25T15:00:00.067 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T15:00:07.050 INFO:teuthology.orchestra.run.trial090.stdout:rbd-mirror-debuginfo-20.2.0-676.gc824a7fa.el9.x 11 MB/s | 79 MB 00:06 2026-02-25T15:00:07.051 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T15:00:07.051 INFO:teuthology.orchestra.run.trial090.stdout:Total 11 MB/s | 79 MB 00:06 2026-02-25T15:00:07.051 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T15:00:07.053 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T15:00:07.053 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T15:00:07.303 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T15:00:07.303 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T15:00:11.428 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T15:00:11.446 INFO:teuthology.orchestra.run.trial090.stdout: Installing : rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T15:00:11.776 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T15:00:11.842 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_ 1/1 2026-02-25T15:00:11.842 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:11.842 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T15:00:11.842 INFO:teuthology.orchestra.run.trial090.stdout: rbd-mirror-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T15:00:11.842 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:11.842 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T15:00:11.875 DEBUG:teuthology.orchestra.run.trial090:> sudo yum -y install rbd-nbd-debuginfo 2026-02-25T15:00:12.193 INFO:teuthology.orchestra.run.trial090.stdout:Last metadata expiration check: 0:39:38 ago on Wed 25 Feb 2026 02:20:34 PM UTC. 2026-02-25T15:00:12.276 INFO:teuthology.orchestra.run.trial090.stdout:Dependencies resolved. 2026-02-25T15:00:12.277 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T15:00:12.277 INFO:teuthology.orchestra.run.trial090.stdout: Package Arch Version Repo Size 2026-02-25T15:00:12.277 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T15:00:12.277 INFO:teuthology.orchestra.run.trial090.stdout:Installing: 2026-02-25T15:00:12.277 INFO:teuthology.orchestra.run.trial090.stdout: rbd-nbd-debuginfo x86_64 2:20.2.0-676.gc824a7fa.el9 ceph 2.4 M 2026-02-25T15:00:12.277 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:12.277 INFO:teuthology.orchestra.run.trial090.stdout:Transaction Summary 2026-02-25T15:00:12.277 INFO:teuthology.orchestra.run.trial090.stdout:================================================================================ 2026-02-25T15:00:12.277 INFO:teuthology.orchestra.run.trial090.stdout:Install 1 Package 2026-02-25T15:00:12.278 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:12.278 INFO:teuthology.orchestra.run.trial090.stdout:Total download size: 2.4 M 2026-02-25T15:00:12.278 INFO:teuthology.orchestra.run.trial090.stdout:Installed size: 12 M 2026-02-25T15:00:12.278 INFO:teuthology.orchestra.run.trial090.stdout:Downloading Packages: 2026-02-25T15:00:13.066 INFO:teuthology.orchestra.run.trial090.stdout:rbd-nbd-debuginfo-20.2.0-676.gc824a7fa.el9.x86_ 3.1 MB/s | 2.4 MB 00:00 2026-02-25T15:00:13.067 INFO:teuthology.orchestra.run.trial090.stdout:-------------------------------------------------------------------------------- 2026-02-25T15:00:13.067 INFO:teuthology.orchestra.run.trial090.stdout:Total 3.1 MB/s | 2.4 MB 00:00 2026-02-25T15:00:13.068 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction check 2026-02-25T15:00:13.069 INFO:teuthology.orchestra.run.trial090.stdout:Transaction check succeeded. 2026-02-25T15:00:13.069 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction test 2026-02-25T15:00:13.093 INFO:teuthology.orchestra.run.trial090.stdout:Transaction test succeeded. 2026-02-25T15:00:13.094 INFO:teuthology.orchestra.run.trial090.stdout:Running transaction 2026-02-25T15:00:13.255 INFO:teuthology.orchestra.run.trial090.stdout: Preparing : 1/1 2026-02-25T15:00:13.273 INFO:teuthology.orchestra.run.trial090.stdout: Installing : rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T15:00:13.345 INFO:teuthology.orchestra.run.trial090.stdout: Running scriptlet: rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T15:00:13.410 INFO:teuthology.orchestra.run.trial090.stdout: Verifying : rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 1/1 2026-02-25T15:00:13.410 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:13.410 INFO:teuthology.orchestra.run.trial090.stdout:Installed: 2026-02-25T15:00:13.410 INFO:teuthology.orchestra.run.trial090.stdout: rbd-nbd-debuginfo-2:20.2.0-676.gc824a7fa.el9.x86_64 2026-02-25T15:00:13.411 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:13.411 INFO:teuthology.orchestra.run.trial090.stdout:Complete! 2026-02-25T15:00:13.444 DEBUG:teuthology.parallel:result is None 2026-02-25T15:00:13.445 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c824a7fa29b067db9ce5a381f59b175918299919 2026-02-25T15:00:13.627 DEBUG:teuthology.orchestra.run.trial090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T15:00:13.649 INFO:teuthology.orchestra.run.trial090.stdout:20.2.0-676.gc824a7fa.el9 2026-02-25T15:00:13.649 INFO:teuthology.packaging:The installed version of ceph is 20.2.0-676.gc824a7fa.el9 2026-02-25T15:00:13.649 INFO:teuthology.task.install:The correct ceph version 20.2.0-676.gc824a7fa is installed. 2026-02-25T15:00:13.650 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c824a7fa29b067db9ce5a381f59b175918299919 2026-02-25T15:00:13.789 DEBUG:teuthology.orchestra.run.trial093:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T15:00:13.821 INFO:teuthology.orchestra.run.trial093.stdout:20.2.0-676.gc824a7fa.el9 2026-02-25T15:00:13.822 INFO:teuthology.packaging:The installed version of ceph is 20.2.0-676.gc824a7fa.el9 2026-02-25T15:00:13.822 INFO:teuthology.task.install:The correct ceph version 20.2.0-676.gc824a7fa is installed. 2026-02-25T15:00:13.825 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c824a7fa29b067db9ce5a381f59b175918299919 2026-02-25T15:00:13.993 DEBUG:teuthology.orchestra.run.trial094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T15:00:14.025 INFO:teuthology.orchestra.run.trial094.stdout:20.2.0-676.gc824a7fa.el9 2026-02-25T15:00:14.025 INFO:teuthology.packaging:The installed version of ceph is 20.2.0-676.gc824a7fa.el9 2026-02-25T15:00:14.025 INFO:teuthology.task.install:The correct ceph version 20.2.0-676.gc824a7fa is installed. 2026-02-25T15:00:14.028 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c824a7fa29b067db9ce5a381f59b175918299919 2026-02-25T15:00:14.182 DEBUG:teuthology.orchestra.run.trial189:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T15:00:14.213 INFO:teuthology.orchestra.run.trial189.stdout:20.2.0-676.gc824a7fa.el9 2026-02-25T15:00:14.213 INFO:teuthology.packaging:The installed version of ceph is 20.2.0-676.gc824a7fa.el9 2026-02-25T15:00:14.213 INFO:teuthology.task.install:The correct ceph version 20.2.0-676.gc824a7fa is installed. 2026-02-25T15:00:14.215 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-25T15:00:14.215 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:14.215 DEBUG:teuthology.orchestra.run.trial090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-25T15:00:14.241 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T15:00:14.241 DEBUG:teuthology.orchestra.run.trial093:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-25T15:00:14.268 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T15:00:14.268 DEBUG:teuthology.orchestra.run.trial094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-25T15:00:14.294 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:14.294 DEBUG:teuthology.orchestra.run.trial189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-25T15:00:14.335 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-25T15:00:14.336 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:14.336 DEBUG:teuthology.orchestra.run.trial090:> sudo dd of=/usr/bin/daemon-helper 2026-02-25T15:00:14.362 DEBUG:teuthology.orchestra.run.trial090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-25T15:00:14.426 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T15:00:14.426 DEBUG:teuthology.orchestra.run.trial093:> sudo dd of=/usr/bin/daemon-helper 2026-02-25T15:00:14.452 DEBUG:teuthology.orchestra.run.trial093:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-25T15:00:14.516 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T15:00:14.516 DEBUG:teuthology.orchestra.run.trial094:> sudo dd of=/usr/bin/daemon-helper 2026-02-25T15:00:14.542 DEBUG:teuthology.orchestra.run.trial094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-25T15:00:14.606 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:14.606 DEBUG:teuthology.orchestra.run.trial189:> sudo dd of=/usr/bin/daemon-helper 2026-02-25T15:00:14.633 DEBUG:teuthology.orchestra.run.trial189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-25T15:00:14.697 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-25T15:00:14.698 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:14.698 DEBUG:teuthology.orchestra.run.trial090:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-25T15:00:14.724 DEBUG:teuthology.orchestra.run.trial090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-25T15:00:14.788 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T15:00:14.788 DEBUG:teuthology.orchestra.run.trial093:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-25T15:00:14.814 DEBUG:teuthology.orchestra.run.trial093:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-25T15:00:14.878 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T15:00:14.878 DEBUG:teuthology.orchestra.run.trial094:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-25T15:00:14.903 DEBUG:teuthology.orchestra.run.trial094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-25T15:00:14.967 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:14.967 DEBUG:teuthology.orchestra.run.trial189:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-25T15:00:14.993 DEBUG:teuthology.orchestra.run.trial189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-25T15:00:15.056 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-25T15:00:15.056 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:15.056 DEBUG:teuthology.orchestra.run.trial090:> sudo dd of=/usr/bin/stdin-killer 2026-02-25T15:00:15.082 DEBUG:teuthology.orchestra.run.trial090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-25T15:00:15.146 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T15:00:15.146 DEBUG:teuthology.orchestra.run.trial093:> sudo dd of=/usr/bin/stdin-killer 2026-02-25T15:00:15.172 DEBUG:teuthology.orchestra.run.trial093:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-25T15:00:15.236 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T15:00:15.236 DEBUG:teuthology.orchestra.run.trial094:> sudo dd of=/usr/bin/stdin-killer 2026-02-25T15:00:15.261 DEBUG:teuthology.orchestra.run.trial094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-25T15:00:15.325 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:15.325 DEBUG:teuthology.orchestra.run.trial189:> sudo dd of=/usr/bin/stdin-killer 2026-02-25T15:00:15.349 DEBUG:teuthology.orchestra.run.trial189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-25T15:00:15.412 INFO:teuthology.run_tasks:Running task ceph... 2026-02-25T15:00:15.485 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-25T15:00:15.485 DEBUG:teuthology.orchestra.run.trial090:> sudo chmod 777 /var/log/ceph 2026-02-25T15:00:15.488 DEBUG:teuthology.orchestra.run.trial093:> sudo chmod 777 /var/log/ceph 2026-02-25T15:00:15.490 DEBUG:teuthology.orchestra.run.trial094:> sudo chmod 777 /var/log/ceph 2026-02-25T15:00:15.492 DEBUG:teuthology.orchestra.run.trial189:> sudo chmod 777 /var/log/ceph 2026-02-25T15:00:15.516 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-25T15:00:15.517 DEBUG:teuthology.orchestra.run.trial090:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-25T15:00:15.554 DEBUG:teuthology.orchestra.run.trial093:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-25T15:00:15.555 DEBUG:teuthology.orchestra.run.trial094:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-25T15:00:15.556 DEBUG:teuthology.orchestra.run.trial189:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-25T15:00:15.579 INFO:tasks.ceph:Creating extra log directories... 2026-02-25T15:00:15.579 DEBUG:teuthology.orchestra.run.trial090:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-25T15:00:15.617 DEBUG:teuthology.orchestra.run.trial093:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-25T15:00:15.619 DEBUG:teuthology.orchestra.run.trial094:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-25T15:00:15.621 DEBUG:teuthology.orchestra.run.trial189:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-25T15:00:15.668 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-25T15:00:15.668 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20, 'rados replica read policy': 'balance'}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 160, 'osd op thread timeout': 150}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'btree', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore onode segment size': 0, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'crush location': 'root=default host=host-$id', 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 240, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request', 'OSD bench result', 'OSD_DOWN'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2026-02-25T15:00:15.669 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209', 'branch': 'tentacle', 'description': 'rados/verify/{centos_latest ceph clusters/fixed-4 d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/{bluestore/{alloc$/{btree} base mem$/{normal-2} onode-segment$/{none} write$/{random/{compr$/{no$/{no}} random}}}} rados read-affinity/balance tasks/rados_api_tests validater/valgrind}', 'email': None, 'first_in_suite': False, 'flavor': 'default', 'job_id': '70209', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'tentacle'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20, 'rados replica read policy': 'balance'}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 160, 'osd op thread timeout': 150}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'btree', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore onode segment size': 0, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'crush location': 'root=default host=host-$id', 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 240, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request', 'OSD bench result', 'OSD_DOWN'], 'mon_bind_msgr2': False, 'sha1': 'c824a7fa29b067db9ce5a381f59b175918299919', 'stop-daemons-timeout': 1200, 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'c824a7fa29b067db9ce5a381f59b175918299919'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'tentacle', 'sha1': 'c824a7fa29b067db9ce5a381f59b175918299919'}}, 'owner': 'scheduled_jayaprakash@soko04.front.sepia.ceph.com', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'], ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'], ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'], ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0']], 'seed': 6877, 'sha1': 'c824a7fa29b067db9ce5a381f59b175918299919', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'tentacle', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_c824a7fa29b067db9ce5a381f59b175918299919/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': 'c824a7fa29b067db9ce5a381f59b175918299919', 'targets': {'trial090.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMifiaRszno3QLj/XgaKwqRtI6u3Q4VHWvtTcCq3FpldrfTPZCTEUyTVwFsaU6JI4yCpRrhz9XyiQHgrp7k5slU=', 'trial093.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHFDe+hnunFXNfvF1s2St2q5un8074tf3AIaIXYMmQE8pu621PsHgx7FlyoYd8dDx03CLtRVk8pQVPVU5Skf1kE=', 'trial094.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA0sFA/J19RVGYTjoHPJti6YO6GdIa4wFWFcZ/quoxS6xXNHCjy6FCTeUJhS3oYG0MXZiIgtxjV19EBBUxMWcBU=', 'trial189.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFsxAT6wDgq/uTugbvXaaIajCMmcpBdDzSZBBOBD9/ujaLxo+HIHmpMuFku+WbwjKfUHYGl2fGoGcQj9GySoufc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'workunit': {'clients': {'client.0': ['rados/test.sh']}, 'env': {'ALLOW_TIMEOUTS': '1'}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["-- Don't run this test on tiny clusters\nif py_len(yaml.roles) < 4 then reject() end\n"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-25_14:08:21', 'tube': 'trial', 'user': 'jayaprakash', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699'} 2026-02-25T15:00:15.669 DEBUG:teuthology.orchestra.run.trial090:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-25T15:00:15.682 DEBUG:teuthology.orchestra.run.trial093:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-25T15:00:15.685 DEBUG:teuthology.orchestra.run.trial094:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-25T15:00:15.687 DEBUG:teuthology.orchestra.run.trial189:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-25T15:00:15.704 DEBUG:teuthology.orchestra.run.trial090:> sudo install -d -m0777 -- /var/run/ceph 2026-02-25T15:00:15.739 DEBUG:teuthology.orchestra.run.trial093:> sudo install -d -m0777 -- /var/run/ceph 2026-02-25T15:00:15.741 DEBUG:teuthology.orchestra.run.trial094:> sudo install -d -m0777 -- /var/run/ceph 2026-02-25T15:00:15.744 DEBUG:teuthology.orchestra.run.trial189:> sudo install -d -m0777 -- /var/run/ceph 2026-02-25T15:00:15.769 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:15.769 DEBUG:teuthology.orchestra.run.trial090:> dd if=/scratch_devs of=/dev/stdout 2026-02-25T15:00:15.817 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-25T15:00:15.817 DEBUG:teuthology.orchestra.run.trial090:> stat /dev/vg_nvme/lv_1 2026-02-25T15:00:15.873 INFO:teuthology.orchestra.run.trial090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-25T15:00:15.874 INFO:teuthology.orchestra.run.trial090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:15.874 INFO:teuthology.orchestra.run.trial090.stdout:Device: 6h/6d Inode: 1304 Links: 1 2026-02-25T15:00:15.874 INFO:teuthology.orchestra.run.trial090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:15.874 INFO:teuthology.orchestra.run.trial090.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:15.874 INFO:teuthology.orchestra.run.trial090.stdout:Access: 2026-02-25 14:22:17.954701445 +0000 2026-02-25T15:00:15.874 INFO:teuthology.orchestra.run.trial090.stdout:Modify: 2026-02-25 14:20:49.782124379 +0000 2026-02-25T15:00:15.874 INFO:teuthology.orchestra.run.trial090.stdout:Change: 2026-02-25 14:20:49.782124379 +0000 2026-02-25T15:00:15.874 INFO:teuthology.orchestra.run.trial090.stdout: Birth: 2026-02-25 14:20:49.782124379 +0000 2026-02-25T15:00:15.875 DEBUG:teuthology.orchestra.run.trial090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-25T15:00:15.937 INFO:teuthology.orchestra.run.trial090.stderr:1+0 records in 2026-02-25T15:00:15.937 INFO:teuthology.orchestra.run.trial090.stderr:1+0 records out 2026-02-25T15:00:15.937 INFO:teuthology.orchestra.run.trial090.stderr:512 bytes copied, 0.000152393 s, 3.4 MB/s 2026-02-25T15:00:15.939 DEBUG:teuthology.orchestra.run.trial090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-25T15:00:15.996 DEBUG:teuthology.orchestra.run.trial090:> stat /dev/vg_nvme/lv_2 2026-02-25T15:00:16.052 INFO:teuthology.orchestra.run.trial090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-25T15:00:16.053 INFO:teuthology.orchestra.run.trial090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:16.053 INFO:teuthology.orchestra.run.trial090.stdout:Device: 6h/6d Inode: 1309 Links: 1 2026-02-25T15:00:16.053 INFO:teuthology.orchestra.run.trial090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:16.053 INFO:teuthology.orchestra.run.trial090.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:16.053 INFO:teuthology.orchestra.run.trial090.stdout:Access: 2026-02-25 14:22:17.955701406 +0000 2026-02-25T15:00:16.053 INFO:teuthology.orchestra.run.trial090.stdout:Modify: 2026-02-25 14:20:49.782124379 +0000 2026-02-25T15:00:16.053 INFO:teuthology.orchestra.run.trial090.stdout:Change: 2026-02-25 14:20:49.782124379 +0000 2026-02-25T15:00:16.053 INFO:teuthology.orchestra.run.trial090.stdout: Birth: 2026-02-25 14:20:49.782124379 +0000 2026-02-25T15:00:16.054 DEBUG:teuthology.orchestra.run.trial090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-25T15:00:16.117 INFO:teuthology.orchestra.run.trial090.stderr:1+0 records in 2026-02-25T15:00:16.117 INFO:teuthology.orchestra.run.trial090.stderr:1+0 records out 2026-02-25T15:00:16.117 INFO:teuthology.orchestra.run.trial090.stderr:512 bytes copied, 0.000142304 s, 3.6 MB/s 2026-02-25T15:00:16.118 DEBUG:teuthology.orchestra.run.trial090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-25T15:00:16.175 DEBUG:teuthology.orchestra.run.trial090:> stat /dev/vg_nvme/lv_3 2026-02-25T15:00:16.231 INFO:teuthology.orchestra.run.trial090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-25T15:00:16.232 INFO:teuthology.orchestra.run.trial090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:16.232 INFO:teuthology.orchestra.run.trial090.stdout:Device: 6h/6d Inode: 1293 Links: 1 2026-02-25T15:00:16.232 INFO:teuthology.orchestra.run.trial090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:16.232 INFO:teuthology.orchestra.run.trial090.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:16.232 INFO:teuthology.orchestra.run.trial090.stdout:Access: 2026-02-25 14:22:17.955701406 +0000 2026-02-25T15:00:16.232 INFO:teuthology.orchestra.run.trial090.stdout:Modify: 2026-02-25 14:20:49.781124418 +0000 2026-02-25T15:00:16.232 INFO:teuthology.orchestra.run.trial090.stdout:Change: 2026-02-25 14:20:49.781124418 +0000 2026-02-25T15:00:16.232 INFO:teuthology.orchestra.run.trial090.stdout: Birth: 2026-02-25 14:20:49.781124418 +0000 2026-02-25T15:00:16.233 DEBUG:teuthology.orchestra.run.trial090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-25T15:00:16.295 INFO:teuthology.orchestra.run.trial090.stderr:1+0 records in 2026-02-25T15:00:16.295 INFO:teuthology.orchestra.run.trial090.stderr:1+0 records out 2026-02-25T15:00:16.295 INFO:teuthology.orchestra.run.trial090.stderr:512 bytes copied, 0.000166008 s, 3.1 MB/s 2026-02-25T15:00:16.296 DEBUG:teuthology.orchestra.run.trial090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-25T15:00:16.353 DEBUG:teuthology.orchestra.run.trial090:> stat /dev/vg_nvme/lv_4 2026-02-25T15:00:16.409 INFO:teuthology.orchestra.run.trial090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-25T15:00:16.410 INFO:teuthology.orchestra.run.trial090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:16.410 INFO:teuthology.orchestra.run.trial090.stdout:Device: 6h/6d Inode: 1288 Links: 1 2026-02-25T15:00:16.410 INFO:teuthology.orchestra.run.trial090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:16.410 INFO:teuthology.orchestra.run.trial090.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:16.410 INFO:teuthology.orchestra.run.trial090.stdout:Access: 2026-02-25 14:22:17.955701406 +0000 2026-02-25T15:00:16.410 INFO:teuthology.orchestra.run.trial090.stdout:Modify: 2026-02-25 14:20:49.781124418 +0000 2026-02-25T15:00:16.410 INFO:teuthology.orchestra.run.trial090.stdout:Change: 2026-02-25 14:20:49.781124418 +0000 2026-02-25T15:00:16.410 INFO:teuthology.orchestra.run.trial090.stdout: Birth: 2026-02-25 14:20:49.781124418 +0000 2026-02-25T15:00:16.411 DEBUG:teuthology.orchestra.run.trial090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-25T15:00:16.472 INFO:teuthology.orchestra.run.trial090.stderr:1+0 records in 2026-02-25T15:00:16.472 INFO:teuthology.orchestra.run.trial090.stderr:1+0 records out 2026-02-25T15:00:16.472 INFO:teuthology.orchestra.run.trial090.stderr:512 bytes copied, 0.000137595 s, 3.7 MB/s 2026-02-25T15:00:16.473 DEBUG:teuthology.orchestra.run.trial090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-25T15:00:16.529 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:16.529 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T15:00:16.529 DEBUG:teuthology.orchestra.run.trial093:> dd if=/scratch_devs of=/dev/stdout 2026-02-25T15:00:16.546 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-25T15:00:16.546 DEBUG:teuthology.orchestra.run.trial093:> stat /dev/vg_nvme/lv_1 2026-02-25T15:00:16.602 INFO:teuthology.orchestra.run.trial093.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-25T15:00:16.602 INFO:teuthology.orchestra.run.trial093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:16.602 INFO:teuthology.orchestra.run.trial093.stdout:Device: 6h/6d Inode: 1295 Links: 1 2026-02-25T15:00:16.602 INFO:teuthology.orchestra.run.trial093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:16.602 INFO:teuthology.orchestra.run.trial093.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:16.602 INFO:teuthology.orchestra.run.trial093.stdout:Access: 2026-02-25 14:21:38.895138327 +0000 2026-02-25T15:00:16.602 INFO:teuthology.orchestra.run.trial093.stdout:Modify: 2026-02-25 14:20:50.854492271 +0000 2026-02-25T15:00:16.602 INFO:teuthology.orchestra.run.trial093.stdout:Change: 2026-02-25 14:20:50.854492271 +0000 2026-02-25T15:00:16.602 INFO:teuthology.orchestra.run.trial093.stdout: Birth: 2026-02-25 14:20:50.854492271 +0000 2026-02-25T15:00:16.603 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-25T15:00:16.664 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records in 2026-02-25T15:00:16.664 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records out 2026-02-25T15:00:16.664 INFO:teuthology.orchestra.run.trial093.stderr:512 bytes copied, 0.000162762 s, 3.1 MB/s 2026-02-25T15:00:16.665 DEBUG:teuthology.orchestra.run.trial093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-25T15:00:16.723 DEBUG:teuthology.orchestra.run.trial093:> stat /dev/vg_nvme/lv_2 2026-02-25T15:00:16.778 INFO:teuthology.orchestra.run.trial093.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-25T15:00:16.778 INFO:teuthology.orchestra.run.trial093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:16.778 INFO:teuthology.orchestra.run.trial093.stdout:Device: 6h/6d Inode: 1293 Links: 1 2026-02-25T15:00:16.778 INFO:teuthology.orchestra.run.trial093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:16.779 INFO:teuthology.orchestra.run.trial093.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:16.779 INFO:teuthology.orchestra.run.trial093.stdout:Access: 2026-02-25 14:21:38.895138327 +0000 2026-02-25T15:00:16.779 INFO:teuthology.orchestra.run.trial093.stdout:Modify: 2026-02-25 14:20:50.854492271 +0000 2026-02-25T15:00:16.779 INFO:teuthology.orchestra.run.trial093.stdout:Change: 2026-02-25 14:20:50.854492271 +0000 2026-02-25T15:00:16.779 INFO:teuthology.orchestra.run.trial093.stdout: Birth: 2026-02-25 14:20:50.854492271 +0000 2026-02-25T15:00:16.779 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-25T15:00:16.840 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records in 2026-02-25T15:00:16.841 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records out 2026-02-25T15:00:16.841 INFO:teuthology.orchestra.run.trial093.stderr:512 bytes copied, 0.000147754 s, 3.5 MB/s 2026-02-25T15:00:16.842 DEBUG:teuthology.orchestra.run.trial093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-25T15:00:16.900 DEBUG:teuthology.orchestra.run.trial093:> stat /dev/vg_nvme/lv_3 2026-02-25T15:00:16.956 INFO:teuthology.orchestra.run.trial093.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-25T15:00:16.956 INFO:teuthology.orchestra.run.trial093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:16.956 INFO:teuthology.orchestra.run.trial093.stdout:Device: 6h/6d Inode: 1310 Links: 1 2026-02-25T15:00:16.956 INFO:teuthology.orchestra.run.trial093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:16.956 INFO:teuthology.orchestra.run.trial093.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:16.957 INFO:teuthology.orchestra.run.trial093.stdout:Access: 2026-02-25 14:21:38.895138327 +0000 2026-02-25T15:00:16.957 INFO:teuthology.orchestra.run.trial093.stdout:Modify: 2026-02-25 14:20:50.855492241 +0000 2026-02-25T15:00:16.957 INFO:teuthology.orchestra.run.trial093.stdout:Change: 2026-02-25 14:20:50.855492241 +0000 2026-02-25T15:00:16.957 INFO:teuthology.orchestra.run.trial093.stdout: Birth: 2026-02-25 14:20:50.855492241 +0000 2026-02-25T15:00:16.957 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-25T15:00:17.018 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records in 2026-02-25T15:00:17.018 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records out 2026-02-25T15:00:17.018 INFO:teuthology.orchestra.run.trial093.stderr:512 bytes copied, 0.000143366 s, 3.6 MB/s 2026-02-25T15:00:17.019 DEBUG:teuthology.orchestra.run.trial093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-25T15:00:17.076 DEBUG:teuthology.orchestra.run.trial093:> stat /dev/vg_nvme/lv_4 2026-02-25T15:00:17.131 INFO:teuthology.orchestra.run.trial093.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-25T15:00:17.131 INFO:teuthology.orchestra.run.trial093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:17.131 INFO:teuthology.orchestra.run.trial093.stdout:Device: 6h/6d Inode: 1287 Links: 1 2026-02-25T15:00:17.131 INFO:teuthology.orchestra.run.trial093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:17.131 INFO:teuthology.orchestra.run.trial093.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:17.132 INFO:teuthology.orchestra.run.trial093.stdout:Access: 2026-02-25 14:21:38.895138327 +0000 2026-02-25T15:00:17.132 INFO:teuthology.orchestra.run.trial093.stdout:Modify: 2026-02-25 14:20:50.853492300 +0000 2026-02-25T15:00:17.132 INFO:teuthology.orchestra.run.trial093.stdout:Change: 2026-02-25 14:20:50.854492271 +0000 2026-02-25T15:00:17.132 INFO:teuthology.orchestra.run.trial093.stdout: Birth: 2026-02-25 14:20:50.853492300 +0000 2026-02-25T15:00:17.132 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-25T15:00:17.193 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records in 2026-02-25T15:00:17.193 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records out 2026-02-25T15:00:17.193 INFO:teuthology.orchestra.run.trial093.stderr:512 bytes copied, 0.000142935 s, 3.6 MB/s 2026-02-25T15:00:17.194 DEBUG:teuthology.orchestra.run.trial093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-25T15:00:17.250 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:17.250 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T15:00:17.250 DEBUG:teuthology.orchestra.run.trial094:> dd if=/scratch_devs of=/dev/stdout 2026-02-25T15:00:17.267 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-25T15:00:17.267 DEBUG:teuthology.orchestra.run.trial094:> stat /dev/vg_nvme/lv_1 2026-02-25T15:00:17.324 INFO:teuthology.orchestra.run.trial094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-25T15:00:17.324 INFO:teuthology.orchestra.run.trial094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:17.324 INFO:teuthology.orchestra.run.trial094.stdout:Device: 6h/6d Inode: 1317 Links: 1 2026-02-25T15:00:17.324 INFO:teuthology.orchestra.run.trial094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:17.324 INFO:teuthology.orchestra.run.trial094.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:17.324 INFO:teuthology.orchestra.run.trial094.stdout:Access: 2026-02-25 14:21:34.340426804 +0000 2026-02-25T15:00:17.324 INFO:teuthology.orchestra.run.trial094.stdout:Modify: 2026-02-25 14:20:47.308244070 +0000 2026-02-25T15:00:17.324 INFO:teuthology.orchestra.run.trial094.stdout:Change: 2026-02-25 14:20:47.308244070 +0000 2026-02-25T15:00:17.324 INFO:teuthology.orchestra.run.trial094.stdout: Birth: 2026-02-25 14:20:47.308244070 +0000 2026-02-25T15:00:17.325 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-25T15:00:17.387 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records in 2026-02-25T15:00:17.388 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records out 2026-02-25T15:00:17.388 INFO:teuthology.orchestra.run.trial094.stderr:512 bytes copied, 0.000250875 s, 2.0 MB/s 2026-02-25T15:00:17.389 DEBUG:teuthology.orchestra.run.trial094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-25T15:00:17.446 DEBUG:teuthology.orchestra.run.trial094:> stat /dev/vg_nvme/lv_2 2026-02-25T15:00:17.503 INFO:teuthology.orchestra.run.trial094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-25T15:00:17.503 INFO:teuthology.orchestra.run.trial094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:17.503 INFO:teuthology.orchestra.run.trial094.stdout:Device: 6h/6d Inode: 1312 Links: 1 2026-02-25T15:00:17.503 INFO:teuthology.orchestra.run.trial094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:17.503 INFO:teuthology.orchestra.run.trial094.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:17.503 INFO:teuthology.orchestra.run.trial094.stdout:Access: 2026-02-25 14:21:34.340426804 +0000 2026-02-25T15:00:17.503 INFO:teuthology.orchestra.run.trial094.stdout:Modify: 2026-02-25 14:20:47.308244070 +0000 2026-02-25T15:00:17.504 INFO:teuthology.orchestra.run.trial094.stdout:Change: 2026-02-25 14:20:47.308244070 +0000 2026-02-25T15:00:17.504 INFO:teuthology.orchestra.run.trial094.stdout: Birth: 2026-02-25 14:20:47.308244070 +0000 2026-02-25T15:00:17.504 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-25T15:00:17.566 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records in 2026-02-25T15:00:17.566 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records out 2026-02-25T15:00:17.566 INFO:teuthology.orchestra.run.trial094.stderr:512 bytes copied, 0.00021623 s, 2.4 MB/s 2026-02-25T15:00:17.567 DEBUG:teuthology.orchestra.run.trial094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-25T15:00:17.623 DEBUG:teuthology.orchestra.run.trial094:> stat /dev/vg_nvme/lv_3 2026-02-25T15:00:17.679 INFO:teuthology.orchestra.run.trial094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-25T15:00:17.679 INFO:teuthology.orchestra.run.trial094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:17.679 INFO:teuthology.orchestra.run.trial094.stdout:Device: 6h/6d Inode: 1306 Links: 1 2026-02-25T15:00:17.679 INFO:teuthology.orchestra.run.trial094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:17.679 INFO:teuthology.orchestra.run.trial094.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:17.680 INFO:teuthology.orchestra.run.trial094.stdout:Access: 2026-02-25 14:21:34.340426804 +0000 2026-02-25T15:00:17.680 INFO:teuthology.orchestra.run.trial094.stdout:Modify: 2026-02-25 14:20:47.307244108 +0000 2026-02-25T15:00:17.680 INFO:teuthology.orchestra.run.trial094.stdout:Change: 2026-02-25 14:20:47.307244108 +0000 2026-02-25T15:00:17.680 INFO:teuthology.orchestra.run.trial094.stdout: Birth: 2026-02-25 14:20:47.307244108 +0000 2026-02-25T15:00:17.680 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-25T15:00:17.741 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records in 2026-02-25T15:00:17.741 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records out 2026-02-25T15:00:17.741 INFO:teuthology.orchestra.run.trial094.stderr:512 bytes copied, 0.000146261 s, 3.5 MB/s 2026-02-25T15:00:17.742 DEBUG:teuthology.orchestra.run.trial094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-25T15:00:17.799 DEBUG:teuthology.orchestra.run.trial094:> stat /dev/vg_nvme/lv_4 2026-02-25T15:00:17.854 INFO:teuthology.orchestra.run.trial094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-25T15:00:17.854 INFO:teuthology.orchestra.run.trial094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:17.854 INFO:teuthology.orchestra.run.trial094.stdout:Device: 6h/6d Inode: 1293 Links: 1 2026-02-25T15:00:17.854 INFO:teuthology.orchestra.run.trial094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:17.854 INFO:teuthology.orchestra.run.trial094.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:17.854 INFO:teuthology.orchestra.run.trial094.stdout:Access: 2026-02-25 14:21:34.340426804 +0000 2026-02-25T15:00:17.854 INFO:teuthology.orchestra.run.trial094.stdout:Modify: 2026-02-25 14:20:47.304244225 +0000 2026-02-25T15:00:17.854 INFO:teuthology.orchestra.run.trial094.stdout:Change: 2026-02-25 14:20:47.305244186 +0000 2026-02-25T15:00:17.854 INFO:teuthology.orchestra.run.trial094.stdout: Birth: 2026-02-25 14:20:47.304244225 +0000 2026-02-25T15:00:17.855 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-25T15:00:17.917 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records in 2026-02-25T15:00:17.917 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records out 2026-02-25T15:00:17.917 INFO:teuthology.orchestra.run.trial094.stderr:512 bytes copied, 0.000136854 s, 3.7 MB/s 2026-02-25T15:00:17.918 DEBUG:teuthology.orchestra.run.trial094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-25T15:00:17.975 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:17.975 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:17.975 DEBUG:teuthology.orchestra.run.trial189:> dd if=/scratch_devs of=/dev/stdout 2026-02-25T15:00:17.992 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-25T15:00:17.992 DEBUG:teuthology.orchestra.run.trial189:> stat /dev/vg_nvme/lv_1 2026-02-25T15:00:18.050 INFO:teuthology.orchestra.run.trial189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-25T15:00:18.050 INFO:teuthology.orchestra.run.trial189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:18.050 INFO:teuthology.orchestra.run.trial189.stdout:Device: 6h/6d Inode: 1290 Links: 1 2026-02-25T15:00:18.050 INFO:teuthology.orchestra.run.trial189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:18.050 INFO:teuthology.orchestra.run.trial189.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:18.050 INFO:teuthology.orchestra.run.trial189.stdout:Access: 2026-02-25 14:22:21.654057613 +0000 2026-02-25T15:00:18.050 INFO:teuthology.orchestra.run.trial189.stdout:Modify: 2026-02-25 14:20:49.573597315 +0000 2026-02-25T15:00:18.050 INFO:teuthology.orchestra.run.trial189.stdout:Change: 2026-02-25 14:20:49.573597315 +0000 2026-02-25T15:00:18.051 INFO:teuthology.orchestra.run.trial189.stdout: Birth: 2026-02-25 14:20:49.573597315 +0000 2026-02-25T15:00:18.051 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-25T15:00:18.112 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records in 2026-02-25T15:00:18.112 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records out 2026-02-25T15:00:18.112 INFO:teuthology.orchestra.run.trial189.stderr:512 bytes copied, 0.000151393 s, 3.4 MB/s 2026-02-25T15:00:18.113 DEBUG:teuthology.orchestra.run.trial189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-25T15:00:18.172 DEBUG:teuthology.orchestra.run.trial189:> stat /dev/vg_nvme/lv_2 2026-02-25T15:00:18.228 INFO:teuthology.orchestra.run.trial189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-25T15:00:18.228 INFO:teuthology.orchestra.run.trial189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:18.228 INFO:teuthology.orchestra.run.trial189.stdout:Device: 6h/6d Inode: 1291 Links: 1 2026-02-25T15:00:18.228 INFO:teuthology.orchestra.run.trial189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:18.229 INFO:teuthology.orchestra.run.trial189.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:18.229 INFO:teuthology.orchestra.run.trial189.stdout:Access: 2026-02-25 14:22:21.654057613 +0000 2026-02-25T15:00:18.229 INFO:teuthology.orchestra.run.trial189.stdout:Modify: 2026-02-25 14:20:49.573597315 +0000 2026-02-25T15:00:18.229 INFO:teuthology.orchestra.run.trial189.stdout:Change: 2026-02-25 14:20:49.573597315 +0000 2026-02-25T15:00:18.229 INFO:teuthology.orchestra.run.trial189.stdout: Birth: 2026-02-25 14:20:49.573597315 +0000 2026-02-25T15:00:18.229 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-25T15:00:18.290 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records in 2026-02-25T15:00:18.290 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records out 2026-02-25T15:00:18.290 INFO:teuthology.orchestra.run.trial189.stderr:512 bytes copied, 0.000146243 s, 3.5 MB/s 2026-02-25T15:00:18.291 DEBUG:teuthology.orchestra.run.trial189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-25T15:00:18.349 DEBUG:teuthology.orchestra.run.trial189:> stat /dev/vg_nvme/lv_3 2026-02-25T15:00:18.405 INFO:teuthology.orchestra.run.trial189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-25T15:00:18.405 INFO:teuthology.orchestra.run.trial189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:18.405 INFO:teuthology.orchestra.run.trial189.stdout:Device: 6h/6d Inode: 1294 Links: 1 2026-02-25T15:00:18.405 INFO:teuthology.orchestra.run.trial189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:18.406 INFO:teuthology.orchestra.run.trial189.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:18.406 INFO:teuthology.orchestra.run.trial189.stdout:Access: 2026-02-25 14:22:21.655057623 +0000 2026-02-25T15:00:18.406 INFO:teuthology.orchestra.run.trial189.stdout:Modify: 2026-02-25 14:20:49.573597315 +0000 2026-02-25T15:00:18.406 INFO:teuthology.orchestra.run.trial189.stdout:Change: 2026-02-25 14:20:49.573597315 +0000 2026-02-25T15:00:18.406 INFO:teuthology.orchestra.run.trial189.stdout: Birth: 2026-02-25 14:20:49.573597315 +0000 2026-02-25T15:00:18.406 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-25T15:00:18.467 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records in 2026-02-25T15:00:18.467 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records out 2026-02-25T15:00:18.467 INFO:teuthology.orchestra.run.trial189.stderr:512 bytes copied, 0.000145261 s, 3.5 MB/s 2026-02-25T15:00:18.469 DEBUG:teuthology.orchestra.run.trial189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-25T15:00:18.526 DEBUG:teuthology.orchestra.run.trial189:> stat /dev/vg_nvme/lv_4 2026-02-25T15:00:18.581 INFO:teuthology.orchestra.run.trial189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-25T15:00:18.581 INFO:teuthology.orchestra.run.trial189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T15:00:18.581 INFO:teuthology.orchestra.run.trial189.stdout:Device: 6h/6d Inode: 1303 Links: 1 2026-02-25T15:00:18.581 INFO:teuthology.orchestra.run.trial189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T15:00:18.581 INFO:teuthology.orchestra.run.trial189.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T15:00:18.581 INFO:teuthology.orchestra.run.trial189.stdout:Access: 2026-02-25 14:22:21.655057623 +0000 2026-02-25T15:00:18.581 INFO:teuthology.orchestra.run.trial189.stdout:Modify: 2026-02-25 14:20:49.573597315 +0000 2026-02-25T15:00:18.581 INFO:teuthology.orchestra.run.trial189.stdout:Change: 2026-02-25 14:20:49.573597315 +0000 2026-02-25T15:00:18.581 INFO:teuthology.orchestra.run.trial189.stdout: Birth: 2026-02-25 14:20:49.573597315 +0000 2026-02-25T15:00:18.581 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-25T15:00:18.644 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records in 2026-02-25T15:00:18.645 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records out 2026-02-25T15:00:18.645 INFO:teuthology.orchestra.run.trial189.stderr:512 bytes copied, 0.000144921 s, 3.5 MB/s 2026-02-25T15:00:18.646 DEBUG:teuthology.orchestra.run.trial189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-25T15:00:18.702 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:18.703 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial090.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial093.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial094.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial189.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2026-02-25T15:00:18.703 INFO:tasks.ceph:Generating config... 2026-02-25T15:00:18.704 INFO:tasks.ceph:[client] debug monc = 20 2026-02-25T15:00:18.704 INFO:tasks.ceph:[client] debug ms = 1 2026-02-25T15:00:18.704 INFO:tasks.ceph:[client] debug objecter = 20 2026-02-25T15:00:18.704 INFO:tasks.ceph:[client] debug rados = 20 2026-02-25T15:00:18.704 INFO:tasks.ceph:[client] rados replica read policy = balance 2026-02-25T15:00:18.704 INFO:tasks.ceph:[global] mon client directed command retry = 5 2026-02-25T15:00:18.704 INFO:tasks.ceph:[global] mon election default strategy = 3 2026-02-25T15:00:18.704 INFO:tasks.ceph:[global] ms bind msgr2 = False 2026-02-25T15:00:18.704 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2026-02-25T15:00:18.704 INFO:tasks.ceph:[global] ms type = async 2026-02-25T15:00:18.704 INFO:tasks.ceph:[global] osd heartbeat grace = 160 2026-02-25T15:00:18.704 INFO:tasks.ceph:[global] osd op thread timeout = 150 2026-02-25T15:00:18.704 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-25T15:00:18.704 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-25T15:00:18.704 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-25T15:00:18.704 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-25T15:00:18.704 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-25T15:00:18.704 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-25T15:00:18.704 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2026-02-25T15:00:18.704 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-25T15:00:18.704 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-25T15:00:18.704 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-25T15:00:18.704 INFO:tasks.ceph:[mon] mon scrub interval = 300 2026-02-25T15:00:18.704 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2026-02-25T15:00:18.704 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-25T15:00:18.704 INFO:tasks.ceph:[osd] bdev async discard = True 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] bdev enable discard = True 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] bluestore allocator = btree 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] bluestore onode segment size = 0 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] bluestore write v2 random = True 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] crush location = root=default host=host-$id 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] debug bluefs = 1 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] debug bluestore = 1 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] debug rocksdb = 10 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] osd class default list = * 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] osd class load list = * 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] osd client watch timeout = 240 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] osd fast shutdown = False 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] osd mclock skip benchmark = True 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] osd op queue = debug_random 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2026-02-25T15:00:18.705 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-25T15:00:18.705 INFO:tasks.ceph:Setting up mon.a... 2026-02-25T15:00:18.705 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-25T15:00:18.741 INFO:teuthology.orchestra.run.trial090.stdout:creating /etc/ceph/ceph.keyring 2026-02-25T15:00:18.744 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-25T15:00:18.819 DEBUG:teuthology.orchestra.run.trial090:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T15:00:18.882 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:10.20.193.90:6789]'), ('mon.b', '[v1:10.20.193.93:6789]'), ('mon.c', '[v1:10.20.193.94:6789]')] 2026-02-25T15:00:18.882 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:10.20.193.90:6789],[v1:10.20.193.93:6789],[v1:10.20.193.94:6789]', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 160, 'osd op thread timeout': 150}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'btree', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore onode segment size': 0, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'crush location': 'root=default host=host-$id', 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 240, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20, 'rados replica read policy': 'balance'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2026-02-25T15:00:18.883 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:18.883 DEBUG:teuthology.orchestra.run.trial090:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-25T15:00:18.938 DEBUG:teuthology.orchestra.run.trial090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:10.20.193.90:6789]' --addv b '[v1:10.20.193.93:6789]' --addv c '[v1:10.20.193.94:6789]' --print /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:19.012 INFO:teuthology.orchestra.run.trial090.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:19.013 INFO:teuthology.orchestra.run.trial090.stdout:monmaptool: generated fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:19.013 INFO:teuthology.orchestra.run.trial090.stdout:setting min_mon_release = tentacle 2026-02-25T15:00:19.013 INFO:teuthology.orchestra.run.trial090.stdout:epoch 0 2026-02-25T15:00:19.013 INFO:teuthology.orchestra.run.trial090.stdout:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:19.013 INFO:teuthology.orchestra.run.trial090.stdout:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:19.013 INFO:teuthology.orchestra.run.trial090.stdout:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:19.013 INFO:teuthology.orchestra.run.trial090.stdout:min_mon_release 20 (tentacle) 2026-02-25T15:00:19.013 INFO:teuthology.orchestra.run.trial090.stdout:election_strategy: 3 2026-02-25T15:00:19.014 INFO:teuthology.orchestra.run.trial090.stdout:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:19.014 INFO:teuthology.orchestra.run.trial090.stdout:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:19.014 INFO:teuthology.orchestra.run.trial090.stdout:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:19.014 INFO:teuthology.orchestra.run.trial090.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2026-02-25T15:00:19.015 DEBUG:teuthology.orchestra.run.trial090:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-25T15:00:19.071 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 63c679b5-d219-4797-8adb-7340f3fd6819... 2026-02-25T15:00:19.073 DEBUG:teuthology.orchestra.run.trial090:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-25T15:00:19.113 DEBUG:teuthology.orchestra.run.trial093:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-25T15:00:19.116 DEBUG:teuthology.orchestra.run.trial094:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-25T15:00:19.118 DEBUG:teuthology.orchestra.run.trial189:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-25T15:00:19.150 INFO:teuthology.orchestra.run.trial090.stdout:[global] 2026-02-25T15:00:19.151 INFO:teuthology.orchestra.run.trial090.stdout: chdir = "" 2026-02-25T15:00:19.151 INFO:teuthology.orchestra.run.trial090.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-25T15:00:19.151 INFO:teuthology.orchestra.run.trial090.stdout: auth supported = cephx 2026-02-25T15:00:19.151 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.151 INFO:teuthology.orchestra.run.trial090.stdout: filestore xattr use omap = true 2026-02-25T15:00:19.151 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.151 INFO:teuthology.orchestra.run.trial090.stdout: mon clock drift allowed = 1.000 2026-02-25T15:00:19.151 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.152 INFO:teuthology.orchestra.run.trial090.stdout: osd crush chooseleaf type = 0 2026-02-25T15:00:19.152 INFO:teuthology.orchestra.run.trial090.stdout: auth debug = true 2026-02-25T15:00:19.152 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.152 INFO:teuthology.orchestra.run.trial090.stdout: ms die on old message = true 2026-02-25T15:00:19.152 INFO:teuthology.orchestra.run.trial090.stdout: ms die on bug = true 2026-02-25T15:00:19.152 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.152 INFO:teuthology.orchestra.run.trial090.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-25T15:00:19.152 INFO:teuthology.orchestra.run.trial090.stdout: mon pg warn max object skew = 0 2026-02-25T15:00:19.152 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.152 INFO:teuthology.orchestra.run.trial090.stdout: # disable pg_autoscaler by default for new pools 2026-02-25T15:00:19.152 INFO:teuthology.orchestra.run.trial090.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-25T15:00:19.152 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.152 INFO:teuthology.orchestra.run.trial090.stdout: osd pool default size = 2 2026-02-25T15:00:19.153 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.153 INFO:teuthology.orchestra.run.trial090.stdout: mon osd allow primary affinity = true 2026-02-25T15:00:19.153 INFO:teuthology.orchestra.run.trial090.stdout: mon osd allow pg remap = true 2026-02-25T15:00:19.153 INFO:teuthology.orchestra.run.trial090.stdout: mon warn on legacy crush tunables = false 2026-02-25T15:00:19.153 INFO:teuthology.orchestra.run.trial090.stdout: mon warn on crush straw calc version zero = false 2026-02-25T15:00:19.153 INFO:teuthology.orchestra.run.trial090.stdout: mon warn on no sortbitwise = false 2026-02-25T15:00:19.153 INFO:teuthology.orchestra.run.trial090.stdout: mon warn on osd down out interval zero = false 2026-02-25T15:00:19.153 INFO:teuthology.orchestra.run.trial090.stdout: mon warn on too few osds = false 2026-02-25T15:00:19.153 INFO:teuthology.orchestra.run.trial090.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-25T15:00:19.153 INFO:teuthology.orchestra.run.trial090.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-25T15:00:19.153 INFO:teuthology.orchestra.run.trial090.stdout: mon_allow_pool_size_one = true 2026-02-25T15:00:19.153 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.154 INFO:teuthology.orchestra.run.trial090.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-25T15:00:19.154 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.154 INFO:teuthology.orchestra.run.trial090.stdout: osd default data pool replay window = 5 2026-02-25T15:00:19.154 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.154 INFO:teuthology.orchestra.run.trial090.stdout: mon allow pool delete = true 2026-02-25T15:00:19.154 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.154 INFO:teuthology.orchestra.run.trial090.stdout: mon cluster log file level = debug 2026-02-25T15:00:19.154 INFO:teuthology.orchestra.run.trial090.stdout: debug asserts on shutdown = true 2026-02-25T15:00:19.154 INFO:teuthology.orchestra.run.trial090.stdout: mon health detail to clog = false 2026-02-25T15:00:19.154 INFO:teuthology.orchestra.run.trial090.stdout: mon host = "[v1:10.20.193.90:6789],[v1:10.20.193.93:6789],[v1:10.20.193.94:6789]" 2026-02-25T15:00:19.154 INFO:teuthology.orchestra.run.trial090.stdout: mon client directed command retry = 5 2026-02-25T15:00:19.154 INFO:teuthology.orchestra.run.trial090.stdout: mon election default strategy = 3 2026-02-25T15:00:19.155 INFO:teuthology.orchestra.run.trial090.stdout: ms bind msgr2 = False 2026-02-25T15:00:19.155 INFO:teuthology.orchestra.run.trial090.stdout: ms inject socket failures = 5000 2026-02-25T15:00:19.155 INFO:teuthology.orchestra.run.trial090.stdout: ms type = async 2026-02-25T15:00:19.155 INFO:teuthology.orchestra.run.trial090.stdout: osd heartbeat grace = 160 2026-02-25T15:00:19.155 INFO:teuthology.orchestra.run.trial090.stdout: osd op thread timeout = 150 2026-02-25T15:00:19.155 INFO:teuthology.orchestra.run.trial090.stdout: fsid = 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:19.155 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.155 INFO:teuthology.orchestra.run.trial090.stdout:[osd] 2026-02-25T15:00:19.155 INFO:teuthology.orchestra.run.trial090.stdout: osd journal size = 100 2026-02-25T15:00:19.155 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.155 INFO:teuthology.orchestra.run.trial090.stdout: osd scrub load threshold = 5.0 2026-02-25T15:00:19.155 INFO:teuthology.orchestra.run.trial090.stdout: osd scrub max interval = 600 2026-02-25T15:00:19.156 INFO:teuthology.orchestra.run.trial090.stdout: osd mclock profile = high_recovery_ops 2026-02-25T15:00:19.156 INFO:teuthology.orchestra.run.trial090.stdout: osd mclock skip benchmark = True 2026-02-25T15:00:19.156 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.156 INFO:teuthology.orchestra.run.trial090.stdout: osd recover clone overlap = true 2026-02-25T15:00:19.156 INFO:teuthology.orchestra.run.trial090.stdout: osd recovery max chunk = 1048576 2026-02-25T15:00:19.156 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.156 INFO:teuthology.orchestra.run.trial090.stdout: osd debug shutdown = true 2026-02-25T15:00:19.156 INFO:teuthology.orchestra.run.trial090.stdout: osd debug op order = true 2026-02-25T15:00:19.156 INFO:teuthology.orchestra.run.trial090.stdout: osd debug verify stray on activate = true 2026-02-25T15:00:19.156 INFO:teuthology.orchestra.run.trial090.stdout: osd debug trim objects = true 2026-02-25T15:00:19.156 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.156 INFO:teuthology.orchestra.run.trial090.stdout: osd open classes on start = true 2026-02-25T15:00:19.156 INFO:teuthology.orchestra.run.trial090.stdout: osd debug pg log writeout = true 2026-02-25T15:00:19.157 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.157 INFO:teuthology.orchestra.run.trial090.stdout: osd deep scrub update digest min age = 30 2026-02-25T15:00:19.157 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.157 INFO:teuthology.orchestra.run.trial090.stdout: osd map max advance = 10 2026-02-25T15:00:19.157 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.157 INFO:teuthology.orchestra.run.trial090.stdout: journal zero on create = true 2026-02-25T15:00:19.157 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.157 INFO:teuthology.orchestra.run.trial090.stdout: filestore ondisk finisher threads = 3 2026-02-25T15:00:19.157 INFO:teuthology.orchestra.run.trial090.stdout: filestore apply finisher threads = 3 2026-02-25T15:00:19.157 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.157 INFO:teuthology.orchestra.run.trial090.stdout: bdev debug aio = true 2026-02-25T15:00:19.157 INFO:teuthology.orchestra.run.trial090.stdout: osd debug misdirected ops = true 2026-02-25T15:00:19.157 INFO:teuthology.orchestra.run.trial090.stdout: bdev async discard = True 2026-02-25T15:00:19.158 INFO:teuthology.orchestra.run.trial090.stdout: bdev enable discard = True 2026-02-25T15:00:19.158 INFO:teuthology.orchestra.run.trial090.stdout: bluestore allocator = btree 2026-02-25T15:00:19.158 INFO:teuthology.orchestra.run.trial090.stdout: bluestore block size = 96636764160 2026-02-25T15:00:19.158 INFO:teuthology.orchestra.run.trial090.stdout: bluestore fsck on mount = True 2026-02-25T15:00:19.158 INFO:teuthology.orchestra.run.trial090.stdout: bluestore onode segment size = 0 2026-02-25T15:00:19.158 INFO:teuthology.orchestra.run.trial090.stdout: bluestore write v2 random = True 2026-02-25T15:00:19.158 INFO:teuthology.orchestra.run.trial090.stdout: bluestore zero block detection = True 2026-02-25T15:00:19.158 INFO:teuthology.orchestra.run.trial090.stdout: crush location = root=default host=host-$id 2026-02-25T15:00:19.158 INFO:teuthology.orchestra.run.trial090.stdout: debug bluefs = 1 2026-02-25T15:00:19.158 INFO:teuthology.orchestra.run.trial090.stdout: debug bluestore = 1 2026-02-25T15:00:19.158 INFO:teuthology.orchestra.run.trial090.stdout: debug monc = 20 2026-02-25T15:00:19.158 INFO:teuthology.orchestra.run.trial090.stdout: debug ms = 1 2026-02-25T15:00:19.159 INFO:teuthology.orchestra.run.trial090.stdout: debug osd = 20 2026-02-25T15:00:19.159 INFO:teuthology.orchestra.run.trial090.stdout: debug rocksdb = 10 2026-02-25T15:00:19.159 INFO:teuthology.orchestra.run.trial090.stdout: mon osd backfillfull_ratio = 0.85 2026-02-25T15:00:19.159 INFO:teuthology.orchestra.run.trial090.stdout: mon osd full ratio = 0.9 2026-02-25T15:00:19.159 INFO:teuthology.orchestra.run.trial090.stdout: mon osd nearfull ratio = 0.8 2026-02-25T15:00:19.159 INFO:teuthology.orchestra.run.trial090.stdout: osd class default list = * 2026-02-25T15:00:19.159 INFO:teuthology.orchestra.run.trial090.stdout: osd class load list = * 2026-02-25T15:00:19.159 INFO:teuthology.orchestra.run.trial090.stdout: osd client watch timeout = 240 2026-02-25T15:00:19.159 INFO:teuthology.orchestra.run.trial090.stdout: osd debug verify cached snaps = True 2026-02-25T15:00:19.159 INFO:teuthology.orchestra.run.trial090.stdout: osd debug verify missing on start = True 2026-02-25T15:00:19.159 INFO:teuthology.orchestra.run.trial090.stdout: osd failsafe full ratio = 0.95 2026-02-25T15:00:19.159 INFO:teuthology.orchestra.run.trial090.stdout: osd fast shutdown = False 2026-02-25T15:00:19.159 INFO:teuthology.orchestra.run.trial090.stdout: osd mclock override recovery settings = True 2026-02-25T15:00:19.159 INFO:teuthology.orchestra.run.trial090.stdout: osd objectstore = bluestore 2026-02-25T15:00:19.160 INFO:teuthology.orchestra.run.trial090.stdout: osd op queue = debug_random 2026-02-25T15:00:19.160 INFO:teuthology.orchestra.run.trial090.stdout: osd op queue cut off = debug_random 2026-02-25T15:00:19.160 INFO:teuthology.orchestra.run.trial090.stdout: osd shutdown pgref assert = True 2026-02-25T15:00:19.160 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.160 INFO:teuthology.orchestra.run.trial090.stdout:[mgr] 2026-02-25T15:00:19.160 INFO:teuthology.orchestra.run.trial090.stdout: debug ms = 1 2026-02-25T15:00:19.160 INFO:teuthology.orchestra.run.trial090.stdout: debug mgr = 20 2026-02-25T15:00:19.160 INFO:teuthology.orchestra.run.trial090.stdout: debug mon = 20 2026-02-25T15:00:19.160 INFO:teuthology.orchestra.run.trial090.stdout: debug auth = 20 2026-02-25T15:00:19.160 INFO:teuthology.orchestra.run.trial090.stdout: mon reweight min pgs per osd = 4 2026-02-25T15:00:19.160 INFO:teuthology.orchestra.run.trial090.stdout: mon reweight min bytes per osd = 10 2026-02-25T15:00:19.160 INFO:teuthology.orchestra.run.trial090.stdout: mgr/telemetry/nag = false 2026-02-25T15:00:19.160 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.161 INFO:teuthology.orchestra.run.trial090.stdout:[mon] 2026-02-25T15:00:19.161 INFO:teuthology.orchestra.run.trial090.stdout: debug ms = 1 2026-02-25T15:00:19.161 INFO:teuthology.orchestra.run.trial090.stdout: debug mon = 20 2026-02-25T15:00:19.161 INFO:teuthology.orchestra.run.trial090.stdout: debug paxos = 20 2026-02-25T15:00:19.161 INFO:teuthology.orchestra.run.trial090.stdout: debug auth = 20 2026-02-25T15:00:19.161 INFO:teuthology.orchestra.run.trial090.stdout: mon data avail warn = 5 2026-02-25T15:00:19.161 INFO:teuthology.orchestra.run.trial090.stdout: mon mgr mkfs grace = 240 2026-02-25T15:00:19.161 INFO:teuthology.orchestra.run.trial090.stdout: mon reweight min pgs per osd = 4 2026-02-25T15:00:19.161 INFO:teuthology.orchestra.run.trial090.stdout: mon osd reporter subtree level = osd 2026-02-25T15:00:19.161 INFO:teuthology.orchestra.run.trial090.stdout: mon osd prime pg temp = true 2026-02-25T15:00:19.161 INFO:teuthology.orchestra.run.trial090.stdout: mon reweight min bytes per osd = 10 2026-02-25T15:00:19.161 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.161 INFO:teuthology.orchestra.run.trial090.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-25T15:00:19.162 INFO:teuthology.orchestra.run.trial090.stdout: auth mon ticket ttl = 660 # 11m 2026-02-25T15:00:19.162 INFO:teuthology.orchestra.run.trial090.stdout: auth service ticket ttl = 240 # 4m 2026-02-25T15:00:19.162 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.162 INFO:teuthology.orchestra.run.trial090.stdout: # don't complain about insecure global_id in the test suite 2026-02-25T15:00:19.162 INFO:teuthology.orchestra.run.trial090.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-25T15:00:19.162 INFO:teuthology.orchestra.run.trial090.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-25T15:00:19.162 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.162 INFO:teuthology.orchestra.run.trial090.stdout: # 1m isn't quite enough 2026-02-25T15:00:19.162 INFO:teuthology.orchestra.run.trial090.stdout: mon_down_mkfs_grace = 2m 2026-02-25T15:00:19.162 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.162 INFO:teuthology.orchestra.run.trial090.stdout: mon_warn_on_filestore_osds = false 2026-02-25T15:00:19.162 INFO:teuthology.orchestra.run.trial090.stdout: mon min osdmap epochs = 50 2026-02-25T15:00:19.163 INFO:teuthology.orchestra.run.trial090.stdout: mon osd crush smoke test = False 2026-02-25T15:00:19.163 INFO:teuthology.orchestra.run.trial090.stdout: mon osdmap full prune interval = 2 2026-02-25T15:00:19.163 INFO:teuthology.orchestra.run.trial090.stdout: mon osdmap full prune min = 15 2026-02-25T15:00:19.163 INFO:teuthology.orchestra.run.trial090.stdout: mon osdmap full prune txsize = 2 2026-02-25T15:00:19.163 INFO:teuthology.orchestra.run.trial090.stdout: mon scrub interval = 300 2026-02-25T15:00:19.163 INFO:teuthology.orchestra.run.trial090.stdout: mon warn on pool no app = False 2026-02-25T15:00:19.163 INFO:teuthology.orchestra.run.trial090.stdout: paxos service trim min = 10 2026-02-25T15:00:19.163 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:00:19.163 INFO:teuthology.orchestra.run.trial090.stdout:[client] 2026-02-25T15:00:19.163 INFO:teuthology.orchestra.run.trial090.stdout: rgw cache enabled = true 2026-02-25T15:00:19.163 INFO:teuthology.orchestra.run.trial090.stdout: rgw enable ops log = true 2026-02-25T15:00:19.163 INFO:teuthology.orchestra.run.trial090.stdout: rgw enable usage log = true 2026-02-25T15:00:19.163 INFO:teuthology.orchestra.run.trial090.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-25T15:00:19.164 INFO:teuthology.orchestra.run.trial090.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-25T15:00:19.164 INFO:teuthology.orchestra.run.trial090.stdout: debug monc = 20 2026-02-25T15:00:19.164 INFO:teuthology.orchestra.run.trial090.stdout: debug ms = 1 2026-02-25T15:00:19.164 INFO:teuthology.orchestra.run.trial090.stdout: debug objecter = 20 2026-02-25T15:00:19.164 INFO:teuthology.orchestra.run.trial090.stdout: debug rados = 20 2026-02-25T15:00:19.164 INFO:teuthology.orchestra.run.trial090.stdout: rados replica read policy = balance 2026-02-25T15:00:19.164 INFO:teuthology.orchestra.run.trial090.stdout:[mon.a] 2026-02-25T15:00:19.164 INFO:teuthology.orchestra.run.trial090.stdout:[mon.b] 2026-02-25T15:00:19.164 INFO:teuthology.orchestra.run.trial090.stdout:[mon.c] 2026-02-25T15:00:19.165 INFO:teuthology.orchestra.run.trial189.stdout:[global] 2026-02-25T15:00:19.165 INFO:teuthology.orchestra.run.trial189.stdout: chdir = "" 2026-02-25T15:00:19.166 INFO:teuthology.orchestra.run.trial189.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-25T15:00:19.166 INFO:teuthology.orchestra.run.trial189.stdout: auth supported = cephx 2026-02-25T15:00:19.166 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.166 INFO:teuthology.orchestra.run.trial189.stdout: filestore xattr use omap = true 2026-02-25T15:00:19.166 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.166 INFO:teuthology.orchestra.run.trial189.stdout: mon clock drift allowed = 1.000 2026-02-25T15:00:19.166 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.166 INFO:teuthology.orchestra.run.trial189.stdout: osd crush chooseleaf type = 0 2026-02-25T15:00:19.166 INFO:teuthology.orchestra.run.trial189.stdout: auth debug = true 2026-02-25T15:00:19.166 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.166 INFO:teuthology.orchestra.run.trial189.stdout: ms die on old message = true 2026-02-25T15:00:19.167 INFO:teuthology.orchestra.run.trial189.stdout: ms die on bug = true 2026-02-25T15:00:19.167 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.167 INFO:teuthology.orchestra.run.trial189.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-25T15:00:19.167 INFO:teuthology.orchestra.run.trial189.stdout: mon pg warn max object skew = 0 2026-02-25T15:00:19.167 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.167 INFO:teuthology.orchestra.run.trial189.stdout: # disable pg_autoscaler by default for new pools 2026-02-25T15:00:19.167 INFO:teuthology.orchestra.run.trial189.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-25T15:00:19.167 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.167 INFO:teuthology.orchestra.run.trial189.stdout: osd pool default size = 2 2026-02-25T15:00:19.167 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.167 INFO:teuthology.orchestra.run.trial189.stdout: mon osd allow primary affinity = true 2026-02-25T15:00:19.167 INFO:teuthology.orchestra.run.trial189.stdout: mon osd allow pg remap = true 2026-02-25T15:00:19.167 INFO:teuthology.orchestra.run.trial189.stdout: mon warn on legacy crush tunables = false 2026-02-25T15:00:19.168 INFO:teuthology.orchestra.run.trial189.stdout: mon warn on crush straw calc version zero = false 2026-02-25T15:00:19.168 INFO:teuthology.orchestra.run.trial189.stdout: mon warn on no sortbitwise = false 2026-02-25T15:00:19.168 INFO:teuthology.orchestra.run.trial189.stdout: mon warn on osd down out interval zero = false 2026-02-25T15:00:19.168 INFO:teuthology.orchestra.run.trial189.stdout: mon warn on too few osds = false 2026-02-25T15:00:19.168 INFO:teuthology.orchestra.run.trial189.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-25T15:00:19.168 INFO:teuthology.orchestra.run.trial189.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-25T15:00:19.168 INFO:teuthology.orchestra.run.trial189.stdout: mon_allow_pool_size_one = true 2026-02-25T15:00:19.168 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.168 INFO:teuthology.orchestra.run.trial189.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-25T15:00:19.168 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.168 INFO:teuthology.orchestra.run.trial189.stdout: osd default data pool replay window = 5 2026-02-25T15:00:19.168 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.169 INFO:teuthology.orchestra.run.trial189.stdout: mon allow pool delete = true 2026-02-25T15:00:19.169 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.169 INFO:teuthology.orchestra.run.trial189.stdout: mon cluster log file level = debug 2026-02-25T15:00:19.169 INFO:teuthology.orchestra.run.trial189.stdout: debug asserts on shutdown = true 2026-02-25T15:00:19.169 INFO:teuthology.orchestra.run.trial189.stdout: mon health detail to clog = false 2026-02-25T15:00:19.169 INFO:teuthology.orchestra.run.trial189.stdout: mon host = "[v1:10.20.193.90:6789],[v1:10.20.193.93:6789],[v1:10.20.193.94:6789]" 2026-02-25T15:00:19.169 INFO:teuthology.orchestra.run.trial189.stdout: mon client directed command retry = 5 2026-02-25T15:00:19.169 INFO:teuthology.orchestra.run.trial189.stdout: mon election default strategy = 3 2026-02-25T15:00:19.169 INFO:teuthology.orchestra.run.trial189.stdout: ms bind msgr2 = False 2026-02-25T15:00:19.169 INFO:teuthology.orchestra.run.trial189.stdout: ms inject socket failures = 5000 2026-02-25T15:00:19.169 INFO:teuthology.orchestra.run.trial189.stdout: ms type = async 2026-02-25T15:00:19.169 INFO:teuthology.orchestra.run.trial189.stdout: osd heartbeat grace = 160 2026-02-25T15:00:19.169 INFO:teuthology.orchestra.run.trial189.stdout: osd op thread timeout = 150 2026-02-25T15:00:19.170 INFO:teuthology.orchestra.run.trial189.stdout: fsid = 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:19.170 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.170 INFO:teuthology.orchestra.run.trial189.stdout:[osd] 2026-02-25T15:00:19.170 INFO:teuthology.orchestra.run.trial189.stdout: osd journal size = 100 2026-02-25T15:00:19.170 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.170 INFO:teuthology.orchestra.run.trial189.stdout: osd scrub load threshold = 5.0 2026-02-25T15:00:19.170 INFO:teuthology.orchestra.run.trial189.stdout: osd scrub max interval = 600 2026-02-25T15:00:19.170 INFO:teuthology.orchestra.run.trial189.stdout: osd mclock profile = high_recovery_ops 2026-02-25T15:00:19.170 INFO:teuthology.orchestra.run.trial189.stdout: osd mclock skip benchmark = True 2026-02-25T15:00:19.170 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.170 INFO:teuthology.orchestra.run.trial189.stdout: osd recover clone overlap = true 2026-02-25T15:00:19.170 INFO:teuthology.orchestra.run.trial189.stdout: osd recovery max chunk = 1048576 2026-02-25T15:00:19.171 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.171 INFO:teuthology.orchestra.run.trial189.stdout: osd debug shutdown = true 2026-02-25T15:00:19.171 INFO:teuthology.orchestra.run.trial189.stdout: osd debug op order = true 2026-02-25T15:00:19.171 INFO:teuthology.orchestra.run.trial189.stdout: osd debug verify stray on activate = true 2026-02-25T15:00:19.171 INFO:teuthology.orchestra.run.trial189.stdout: osd debug trim objects = true 2026-02-25T15:00:19.171 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.171 INFO:teuthology.orchestra.run.trial189.stdout: osd open classes on start = true 2026-02-25T15:00:19.171 INFO:teuthology.orchestra.run.trial189.stdout: osd debug pg log writeout = true 2026-02-25T15:00:19.171 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.171 INFO:teuthology.orchestra.run.trial189.stdout: osd deep scrub update digest min age = 30 2026-02-25T15:00:19.171 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.171 INFO:teuthology.orchestra.run.trial189.stdout: osd map max advance = 10 2026-02-25T15:00:19.171 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.172 INFO:teuthology.orchestra.run.trial189.stdout: journal zero on create = true 2026-02-25T15:00:19.172 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.172 INFO:teuthology.orchestra.run.trial189.stdout: filestore ondisk finisher threads = 3 2026-02-25T15:00:19.172 INFO:teuthology.orchestra.run.trial189.stdout: filestore apply finisher threads = 3 2026-02-25T15:00:19.172 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.172 INFO:teuthology.orchestra.run.trial189.stdout: bdev debug aio = true 2026-02-25T15:00:19.172 INFO:teuthology.orchestra.run.trial189.stdout: osd debug misdirected ops = true 2026-02-25T15:00:19.172 INFO:teuthology.orchestra.run.trial189.stdout: bdev async discard = True 2026-02-25T15:00:19.172 INFO:teuthology.orchestra.run.trial189.stdout: bdev enable discard = True 2026-02-25T15:00:19.172 INFO:teuthology.orchestra.run.trial189.stdout: bluestore allocator = btree 2026-02-25T15:00:19.172 INFO:teuthology.orchestra.run.trial189.stdout: bluestore block size = 96636764160 2026-02-25T15:00:19.172 INFO:teuthology.orchestra.run.trial189.stdout: bluestore fsck on mount = True 2026-02-25T15:00:19.173 INFO:teuthology.orchestra.run.trial189.stdout: bluestore onode segment size = 0 2026-02-25T15:00:19.173 INFO:teuthology.orchestra.run.trial189.stdout: bluestore write v2 random = True 2026-02-25T15:00:19.173 INFO:teuthology.orchestra.run.trial189.stdout: bluestore zero block detection = True 2026-02-25T15:00:19.173 INFO:teuthology.orchestra.run.trial189.stdout: crush location = root=default host=host-$id 2026-02-25T15:00:19.173 INFO:teuthology.orchestra.run.trial189.stdout: debug bluefs = 1 2026-02-25T15:00:19.173 INFO:teuthology.orchestra.run.trial189.stdout: debug bluestore = 1 2026-02-25T15:00:19.173 INFO:teuthology.orchestra.run.trial189.stdout: debug monc = 20 2026-02-25T15:00:19.173 INFO:teuthology.orchestra.run.trial189.stdout: debug ms = 1 2026-02-25T15:00:19.173 INFO:teuthology.orchestra.run.trial189.stdout: debug osd = 20 2026-02-25T15:00:19.173 INFO:teuthology.orchestra.run.trial189.stdout: debug rocksdb = 10 2026-02-25T15:00:19.173 INFO:teuthology.orchestra.run.trial189.stdout: mon osd backfillfull_ratio = 0.85 2026-02-25T15:00:19.173 INFO:teuthology.orchestra.run.trial189.stdout: mon osd full ratio = 0.9 2026-02-25T15:00:19.173 INFO:teuthology.orchestra.run.trial189.stdout: mon osd nearfull ratio = 0.8 2026-02-25T15:00:19.174 INFO:teuthology.orchestra.run.trial189.stdout: osd class default list = * 2026-02-25T15:00:19.174 INFO:teuthology.orchestra.run.trial189.stdout: osd class load list = * 2026-02-25T15:00:19.174 INFO:teuthology.orchestra.run.trial189.stdout: osd client watch timeout = 240 2026-02-25T15:00:19.174 INFO:teuthology.orchestra.run.trial189.stdout: osd debug verify cached snaps = True 2026-02-25T15:00:19.174 INFO:teuthology.orchestra.run.trial189.stdout: osd debug verify missing on start = True 2026-02-25T15:00:19.174 INFO:teuthology.orchestra.run.trial189.stdout: osd failsafe full ratio = 0.95 2026-02-25T15:00:19.174 INFO:teuthology.orchestra.run.trial189.stdout: osd fast shutdown = False 2026-02-25T15:00:19.174 INFO:teuthology.orchestra.run.trial189.stdout: osd mclock override recovery settings = True 2026-02-25T15:00:19.174 INFO:teuthology.orchestra.run.trial189.stdout: osd objectstore = bluestore 2026-02-25T15:00:19.174 INFO:teuthology.orchestra.run.trial189.stdout: osd op queue = debug_random 2026-02-25T15:00:19.174 INFO:teuthology.orchestra.run.trial189.stdout: osd op queue cut off = debug_random 2026-02-25T15:00:19.174 INFO:teuthology.orchestra.run.trial189.stdout: osd shutdown pgref assert = True 2026-02-25T15:00:19.174 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.175 INFO:teuthology.orchestra.run.trial189.stdout:[mgr] 2026-02-25T15:00:19.175 INFO:teuthology.orchestra.run.trial189.stdout: debug ms = 1 2026-02-25T15:00:19.175 INFO:teuthology.orchestra.run.trial189.stdout: debug mgr = 20 2026-02-25T15:00:19.175 INFO:teuthology.orchestra.run.trial189.stdout: debug mon = 20 2026-02-25T15:00:19.175 INFO:teuthology.orchestra.run.trial189.stdout: debug auth = 20 2026-02-25T15:00:19.175 INFO:teuthology.orchestra.run.trial189.stdout: mon reweight min pgs per osd = 4 2026-02-25T15:00:19.175 INFO:teuthology.orchestra.run.trial189.stdout: mon reweight min bytes per osd = 10 2026-02-25T15:00:19.175 INFO:teuthology.orchestra.run.trial189.stdout: mgr/telemetry/nag = false 2026-02-25T15:00:19.175 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.175 INFO:teuthology.orchestra.run.trial189.stdout:[mon] 2026-02-25T15:00:19.175 INFO:teuthology.orchestra.run.trial189.stdout: debug ms = 1 2026-02-25T15:00:19.175 INFO:teuthology.orchestra.run.trial189.stdout: debug mon = 20 2026-02-25T15:00:19.176 INFO:teuthology.orchestra.run.trial189.stdout: debug paxos = 20 2026-02-25T15:00:19.176 INFO:teuthology.orchestra.run.trial189.stdout: debug auth = 20 2026-02-25T15:00:19.176 INFO:teuthology.orchestra.run.trial189.stdout: mon data avail warn = 5 2026-02-25T15:00:19.176 INFO:teuthology.orchestra.run.trial189.stdout: mon mgr mkfs grace = 240 2026-02-25T15:00:19.176 INFO:teuthology.orchestra.run.trial189.stdout: mon reweight min pgs per osd = 4 2026-02-25T15:00:19.176 INFO:teuthology.orchestra.run.trial189.stdout: mon osd reporter subtree level = osd 2026-02-25T15:00:19.176 INFO:teuthology.orchestra.run.trial189.stdout: mon osd prime pg temp = true 2026-02-25T15:00:19.176 INFO:teuthology.orchestra.run.trial189.stdout: mon reweight min bytes per osd = 10 2026-02-25T15:00:19.176 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.176 INFO:teuthology.orchestra.run.trial189.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-25T15:00:19.176 INFO:teuthology.orchestra.run.trial189.stdout: auth mon ticket ttl = 660 # 11m 2026-02-25T15:00:19.176 INFO:teuthology.orchestra.run.trial189.stdout: auth service ticket ttl = 240 # 4m 2026-02-25T15:00:19.176 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.177 INFO:teuthology.orchestra.run.trial189.stdout: # don't complain about insecure global_id in the test suite 2026-02-25T15:00:19.177 INFO:teuthology.orchestra.run.trial189.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-25T15:00:19.177 INFO:teuthology.orchestra.run.trial189.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-25T15:00:19.177 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.177 INFO:teuthology.orchestra.run.trial189.stdout: # 1m isn't quite enough 2026-02-25T15:00:19.177 INFO:teuthology.orchestra.run.trial189.stdout: mon_down_mkfs_grace = 2m 2026-02-25T15:00:19.177 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.177 INFO:teuthology.orchestra.run.trial189.stdout: mon_warn_on_filestore_osds = false 2026-02-25T15:00:19.177 INFO:teuthology.orchestra.run.trial189.stdout: mon min osdmap epochs = 50 2026-02-25T15:00:19.177 INFO:teuthology.orchestra.run.trial189.stdout: mon osd crush smoke test = False 2026-02-25T15:00:19.177 INFO:teuthology.orchestra.run.trial189.stdout: mon osdmap full prune interval = 2 2026-02-25T15:00:19.177 INFO:teuthology.orchestra.run.trial189.stdout: mon osdmap full prune min = 15 2026-02-25T15:00:19.177 INFO:teuthology.orchestra.run.trial189.stdout: mon osdmap full prune txsize = 2 2026-02-25T15:00:19.178 INFO:teuthology.orchestra.run.trial189.stdout: mon scrub interval = 300 2026-02-25T15:00:19.178 INFO:teuthology.orchestra.run.trial189.stdout: mon warn on pool no app = False 2026-02-25T15:00:19.178 INFO:teuthology.orchestra.run.trial189.stdout: paxos service trim min = 10 2026-02-25T15:00:19.178 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-25T15:00:19.178 INFO:teuthology.orchestra.run.trial189.stdout:[client] 2026-02-25T15:00:19.178 INFO:teuthology.orchestra.run.trial189.stdout: rgw cache enabled = true 2026-02-25T15:00:19.178 INFO:teuthology.orchestra.run.trial189.stdout: rgw enable ops log = true 2026-02-25T15:00:19.178 INFO:teuthology.orchestra.run.trial189.stdout: rgw enable usage log = true 2026-02-25T15:00:19.178 INFO:teuthology.orchestra.run.trial189.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-25T15:00:19.178 INFO:teuthology.orchestra.run.trial189.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-25T15:00:19.178 INFO:teuthology.orchestra.run.trial189.stdout: debug monc = 20 2026-02-25T15:00:19.178 INFO:teuthology.orchestra.run.trial189.stdout: debug ms = 1 2026-02-25T15:00:19.178 INFO:teuthology.orchestra.run.trial189.stdout: debug objecter = 20 2026-02-25T15:00:19.179 INFO:teuthology.orchestra.run.trial189.stdout: debug rados = 20 2026-02-25T15:00:19.179 INFO:teuthology.orchestra.run.trial189.stdout: rados replica read policy = balance 2026-02-25T15:00:19.179 INFO:teuthology.orchestra.run.trial189.stdout:[mon.a] 2026-02-25T15:00:19.179 INFO:teuthology.orchestra.run.trial189.stdout:[mon.b] 2026-02-25T15:00:19.179 INFO:teuthology.orchestra.run.trial189.stdout:[mon.c] 2026-02-25T15:00:19.179 INFO:teuthology.orchestra.run.trial094.stdout:[global] 2026-02-25T15:00:19.179 INFO:teuthology.orchestra.run.trial094.stdout: chdir = "" 2026-02-25T15:00:19.179 INFO:teuthology.orchestra.run.trial094.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-25T15:00:19.179 INFO:teuthology.orchestra.run.trial094.stdout: auth supported = cephx 2026-02-25T15:00:19.179 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.179 INFO:teuthology.orchestra.run.trial094.stdout: filestore xattr use omap = true 2026-02-25T15:00:19.180 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.180 INFO:teuthology.orchestra.run.trial094.stdout: mon clock drift allowed = 1.000 2026-02-25T15:00:19.180 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.180 INFO:teuthology.orchestra.run.trial094.stdout: osd crush chooseleaf type = 0 2026-02-25T15:00:19.180 INFO:teuthology.orchestra.run.trial094.stdout: auth debug = true 2026-02-25T15:00:19.180 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.180 INFO:teuthology.orchestra.run.trial094.stdout: ms die on old message = true 2026-02-25T15:00:19.180 INFO:teuthology.orchestra.run.trial094.stdout: ms die on bug = true 2026-02-25T15:00:19.180 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.180 INFO:teuthology.orchestra.run.trial094.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-25T15:00:19.180 INFO:teuthology.orchestra.run.trial094.stdout: mon pg warn max object skew = 0 2026-02-25T15:00:19.180 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.181 INFO:teuthology.orchestra.run.trial094.stdout: # disable pg_autoscaler by default for new pools 2026-02-25T15:00:19.181 INFO:teuthology.orchestra.run.trial094.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-25T15:00:19.181 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.181 INFO:teuthology.orchestra.run.trial094.stdout: osd pool default size = 2 2026-02-25T15:00:19.181 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.181 INFO:teuthology.orchestra.run.trial094.stdout: mon osd allow primary affinity = true 2026-02-25T15:00:19.181 INFO:teuthology.orchestra.run.trial094.stdout: mon osd allow pg remap = true 2026-02-25T15:00:19.181 INFO:teuthology.orchestra.run.trial094.stdout: mon warn on legacy crush tunables = false 2026-02-25T15:00:19.181 INFO:teuthology.orchestra.run.trial094.stdout: mon warn on crush straw calc version zero = false 2026-02-25T15:00:19.181 INFO:teuthology.orchestra.run.trial094.stdout: mon warn on no sortbitwise = false 2026-02-25T15:00:19.181 INFO:teuthology.orchestra.run.trial094.stdout: mon warn on osd down out interval zero = false 2026-02-25T15:00:19.181 INFO:teuthology.orchestra.run.trial094.stdout: mon warn on too few osds = false 2026-02-25T15:00:19.182 INFO:teuthology.orchestra.run.trial094.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-25T15:00:19.182 INFO:teuthology.orchestra.run.trial094.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-25T15:00:19.182 INFO:teuthology.orchestra.run.trial094.stdout: mon_allow_pool_size_one = true 2026-02-25T15:00:19.182 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.182 INFO:teuthology.orchestra.run.trial094.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-25T15:00:19.182 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.182 INFO:teuthology.orchestra.run.trial094.stdout: osd default data pool replay window = 5 2026-02-25T15:00:19.182 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.182 INFO:teuthology.orchestra.run.trial094.stdout: mon allow pool delete = true 2026-02-25T15:00:19.182 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.182 INFO:teuthology.orchestra.run.trial094.stdout: mon cluster log file level = debug 2026-02-25T15:00:19.182 INFO:teuthology.orchestra.run.trial094.stdout: debug asserts on shutdown = true 2026-02-25T15:00:19.182 INFO:teuthology.orchestra.run.trial094.stdout: mon health detail to clog = false 2026-02-25T15:00:19.183 INFO:teuthology.orchestra.run.trial094.stdout: mon host = "[v1:10.20.193.90:6789],[v1:10.20.193.93:6789],[v1:10.20.193.94:6789]" 2026-02-25T15:00:19.183 INFO:teuthology.orchestra.run.trial094.stdout: mon client directed command retry = 5 2026-02-25T15:00:19.183 INFO:teuthology.orchestra.run.trial094.stdout: mon election default strategy = 3 2026-02-25T15:00:19.183 INFO:teuthology.orchestra.run.trial094.stdout: ms bind msgr2 = False 2026-02-25T15:00:19.183 INFO:teuthology.orchestra.run.trial094.stdout: ms inject socket failures = 5000 2026-02-25T15:00:19.183 INFO:teuthology.orchestra.run.trial094.stdout: ms type = async 2026-02-25T15:00:19.183 INFO:teuthology.orchestra.run.trial094.stdout: osd heartbeat grace = 160 2026-02-25T15:00:19.183 INFO:teuthology.orchestra.run.trial094.stdout: osd op thread timeout = 150 2026-02-25T15:00:19.183 INFO:teuthology.orchestra.run.trial094.stdout: fsid = 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:19.183 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.183 INFO:teuthology.orchestra.run.trial094.stdout:[osd] 2026-02-25T15:00:19.183 INFO:teuthology.orchestra.run.trial094.stdout: osd journal size = 100 2026-02-25T15:00:19.183 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.183 INFO:teuthology.orchestra.run.trial094.stdout: osd scrub load threshold = 5.0 2026-02-25T15:00:19.184 INFO:teuthology.orchestra.run.trial094.stdout: osd scrub max interval = 600 2026-02-25T15:00:19.184 INFO:teuthology.orchestra.run.trial094.stdout: osd mclock profile = high_recovery_ops 2026-02-25T15:00:19.184 INFO:teuthology.orchestra.run.trial094.stdout: osd mclock skip benchmark = True 2026-02-25T15:00:19.184 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.184 INFO:teuthology.orchestra.run.trial094.stdout: osd recover clone overlap = true 2026-02-25T15:00:19.184 INFO:teuthology.orchestra.run.trial094.stdout: osd recovery max chunk = 1048576 2026-02-25T15:00:19.184 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.184 INFO:teuthology.orchestra.run.trial094.stdout: osd debug shutdown = true 2026-02-25T15:00:19.184 INFO:teuthology.orchestra.run.trial094.stdout: osd debug op order = true 2026-02-25T15:00:19.184 INFO:teuthology.orchestra.run.trial094.stdout: osd debug verify stray on activate = true 2026-02-25T15:00:19.184 INFO:teuthology.orchestra.run.trial094.stdout: osd debug trim objects = true 2026-02-25T15:00:19.184 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.184 INFO:teuthology.orchestra.run.trial094.stdout: osd open classes on start = true 2026-02-25T15:00:19.184 INFO:teuthology.orchestra.run.trial094.stdout: osd debug pg log writeout = true 2026-02-25T15:00:19.185 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.185 INFO:teuthology.orchestra.run.trial094.stdout: osd deep scrub update digest min age = 30 2026-02-25T15:00:19.185 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.185 INFO:teuthology.orchestra.run.trial094.stdout: osd map max advance = 10 2026-02-25T15:00:19.185 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.185 INFO:teuthology.orchestra.run.trial094.stdout: journal zero on create = true 2026-02-25T15:00:19.185 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.185 INFO:teuthology.orchestra.run.trial094.stdout: filestore ondisk finisher threads = 3 2026-02-25T15:00:19.185 INFO:teuthology.orchestra.run.trial094.stdout: filestore apply finisher threads = 3 2026-02-25T15:00:19.185 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.185 INFO:teuthology.orchestra.run.trial094.stdout: bdev debug aio = true 2026-02-25T15:00:19.185 INFO:teuthology.orchestra.run.trial094.stdout: osd debug misdirected ops = true 2026-02-25T15:00:19.185 INFO:teuthology.orchestra.run.trial094.stdout: bdev async discard = True 2026-02-25T15:00:19.185 INFO:teuthology.orchestra.run.trial094.stdout: bdev enable discard = True 2026-02-25T15:00:19.186 INFO:teuthology.orchestra.run.trial094.stdout: bluestore allocator = btree 2026-02-25T15:00:19.186 INFO:teuthology.orchestra.run.trial094.stdout: bluestore block size = 96636764160 2026-02-25T15:00:19.186 INFO:teuthology.orchestra.run.trial094.stdout: bluestore fsck on mount = True 2026-02-25T15:00:19.186 INFO:teuthology.orchestra.run.trial094.stdout: bluestore onode segment size = 0 2026-02-25T15:00:19.186 INFO:teuthology.orchestra.run.trial094.stdout: bluestore write v2 random = True 2026-02-25T15:00:19.186 INFO:teuthology.orchestra.run.trial094.stdout: bluestore zero block detection = True 2026-02-25T15:00:19.186 INFO:teuthology.orchestra.run.trial094.stdout: crush location = root=default host=host-$id 2026-02-25T15:00:19.186 INFO:teuthology.orchestra.run.trial094.stdout: debug bluefs = 1 2026-02-25T15:00:19.186 INFO:teuthology.orchestra.run.trial094.stdout: debug bluestore = 1 2026-02-25T15:00:19.186 INFO:teuthology.orchestra.run.trial094.stdout: debug monc = 20 2026-02-25T15:00:19.186 INFO:teuthology.orchestra.run.trial094.stdout: debug ms = 1 2026-02-25T15:00:19.186 INFO:teuthology.orchestra.run.trial094.stdout: debug osd = 20 2026-02-25T15:00:19.186 INFO:teuthology.orchestra.run.trial094.stdout: debug rocksdb = 10 2026-02-25T15:00:19.186 INFO:teuthology.orchestra.run.trial094.stdout: mon osd backfillfull_ratio = 0.85 2026-02-25T15:00:19.186 INFO:teuthology.orchestra.run.trial094.stdout: mon osd full ratio = 0.9 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout: mon osd nearfull ratio = 0.8 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout: osd class default list = * 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout: osd class load list = * 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout: osd client watch timeout = 240 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout: osd debug verify cached snaps = True 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout: osd debug verify missing on start = True 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout: osd failsafe full ratio = 0.95 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout: osd fast shutdown = False 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout: osd mclock override recovery settings = True 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout: osd objectstore = bluestore 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout: osd op queue = debug_random 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout: osd op queue cut off = debug_random 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout: osd shutdown pgref assert = True 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout:[mgr] 2026-02-25T15:00:19.187 INFO:teuthology.orchestra.run.trial094.stdout: debug ms = 1 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout: debug mgr = 20 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout: debug mon = 20 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout: debug auth = 20 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout: mon reweight min pgs per osd = 4 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout: mon reweight min bytes per osd = 10 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout: mgr/telemetry/nag = false 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout:[mon] 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout: debug ms = 1 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout: debug mon = 20 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout: debug paxos = 20 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout: debug auth = 20 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout: mon data avail warn = 5 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout: mon mgr mkfs grace = 240 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout: mon reweight min pgs per osd = 4 2026-02-25T15:00:19.188 INFO:teuthology.orchestra.run.trial094.stdout: mon osd reporter subtree level = osd 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: mon osd prime pg temp = true 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: mon reweight min bytes per osd = 10 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: auth mon ticket ttl = 660 # 11m 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: auth service ticket ttl = 240 # 4m 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: # don't complain about insecure global_id in the test suite 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: # 1m isn't quite enough 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: mon_down_mkfs_grace = 2m 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: mon_warn_on_filestore_osds = false 2026-02-25T15:00:19.189 INFO:teuthology.orchestra.run.trial094.stdout: mon min osdmap epochs = 50 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: mon osd crush smoke test = False 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: mon osdmap full prune interval = 2 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: mon osdmap full prune min = 15 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: mon osdmap full prune txsize = 2 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: mon scrub interval = 300 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: mon warn on pool no app = False 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: paxos service trim min = 10 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout:[client] 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: rgw cache enabled = true 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: rgw enable ops log = true 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: rgw enable usage log = true 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: debug monc = 20 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: debug ms = 1 2026-02-25T15:00:19.190 INFO:teuthology.orchestra.run.trial094.stdout: debug objecter = 20 2026-02-25T15:00:19.191 INFO:teuthology.orchestra.run.trial094.stdout: debug rados = 20 2026-02-25T15:00:19.191 INFO:teuthology.orchestra.run.trial094.stdout: rados replica read policy = balance 2026-02-25T15:00:19.191 INFO:teuthology.orchestra.run.trial094.stdout:[mon.a] 2026-02-25T15:00:19.191 INFO:teuthology.orchestra.run.trial094.stdout:[mon.b] 2026-02-25T15:00:19.191 INFO:teuthology.orchestra.run.trial094.stdout:[mon.c] 2026-02-25T15:00:19.192 INFO:teuthology.orchestra.run.trial093.stdout:[global] 2026-02-25T15:00:19.192 INFO:teuthology.orchestra.run.trial093.stdout: chdir = "" 2026-02-25T15:00:19.192 INFO:teuthology.orchestra.run.trial093.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-25T15:00:19.192 INFO:teuthology.orchestra.run.trial093.stdout: auth supported = cephx 2026-02-25T15:00:19.192 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: filestore xattr use omap = true 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: mon clock drift allowed = 1.000 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: osd crush chooseleaf type = 0 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: auth debug = true 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: ms die on old message = true 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: ms die on bug = true 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: mon pg warn max object skew = 0 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: # disable pg_autoscaler by default for new pools 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: osd pool default size = 2 2026-02-25T15:00:19.193 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: mon osd allow primary affinity = true 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: mon osd allow pg remap = true 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: mon warn on legacy crush tunables = false 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: mon warn on crush straw calc version zero = false 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: mon warn on no sortbitwise = false 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: mon warn on osd down out interval zero = false 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: mon warn on too few osds = false 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: mon_allow_pool_size_one = true 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: osd default data pool replay window = 5 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: mon allow pool delete = true 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: mon cluster log file level = debug 2026-02-25T15:00:19.194 INFO:teuthology.orchestra.run.trial093.stdout: debug asserts on shutdown = true 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: mon health detail to clog = false 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: mon host = "[v1:10.20.193.90:6789],[v1:10.20.193.93:6789],[v1:10.20.193.94:6789]" 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: mon client directed command retry = 5 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: mon election default strategy = 3 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: ms bind msgr2 = False 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: ms inject socket failures = 5000 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: ms type = async 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: osd heartbeat grace = 160 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: osd op thread timeout = 150 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: fsid = 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout:[osd] 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: osd journal size = 100 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: osd scrub load threshold = 5.0 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: osd scrub max interval = 600 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: osd mclock profile = high_recovery_ops 2026-02-25T15:00:19.195 INFO:teuthology.orchestra.run.trial093.stdout: osd mclock skip benchmark = True 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: osd recover clone overlap = true 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: osd recovery max chunk = 1048576 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: osd debug shutdown = true 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: osd debug op order = true 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: osd debug verify stray on activate = true 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: osd debug trim objects = true 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: osd open classes on start = true 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: osd debug pg log writeout = true 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: osd deep scrub update digest min age = 30 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: osd map max advance = 10 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: journal zero on create = true 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: filestore ondisk finisher threads = 3 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: filestore apply finisher threads = 3 2026-02-25T15:00:19.196 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: bdev debug aio = true 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: osd debug misdirected ops = true 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: bdev async discard = True 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: bdev enable discard = True 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: bluestore allocator = btree 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: bluestore block size = 96636764160 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: bluestore fsck on mount = True 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: bluestore onode segment size = 0 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: bluestore write v2 random = True 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: bluestore zero block detection = True 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: crush location = root=default host=host-$id 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: debug bluefs = 1 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: debug bluestore = 1 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: debug monc = 20 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: debug ms = 1 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: debug osd = 20 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: debug rocksdb = 10 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: mon osd backfillfull_ratio = 0.85 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: mon osd full ratio = 0.9 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: mon osd nearfull ratio = 0.8 2026-02-25T15:00:19.197 INFO:teuthology.orchestra.run.trial093.stdout: osd class default list = * 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: osd class load list = * 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: osd client watch timeout = 240 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: osd debug verify cached snaps = True 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: osd debug verify missing on start = True 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: osd failsafe full ratio = 0.95 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: osd fast shutdown = False 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: osd mclock override recovery settings = True 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: osd objectstore = bluestore 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: osd op queue = debug_random 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: osd op queue cut off = debug_random 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: osd shutdown pgref assert = True 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout:[mgr] 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: debug ms = 1 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: debug mgr = 20 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: debug mon = 20 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: debug auth = 20 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: mon reweight min pgs per osd = 4 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: mon reweight min bytes per osd = 10 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: mgr/telemetry/nag = false 2026-02-25T15:00:19.198 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout:[mon] 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: debug ms = 1 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: debug mon = 20 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: debug paxos = 20 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: debug auth = 20 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: mon data avail warn = 5 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: mon mgr mkfs grace = 240 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: mon reweight min pgs per osd = 4 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: mon osd reporter subtree level = osd 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: mon osd prime pg temp = true 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: mon reweight min bytes per osd = 10 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: auth mon ticket ttl = 660 # 11m 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: auth service ticket ttl = 240 # 4m 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: # don't complain about insecure global_id in the test suite 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: # 1m isn't quite enough 2026-02-25T15:00:19.199 INFO:teuthology.orchestra.run.trial093.stdout: mon_down_mkfs_grace = 2m 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: mon_warn_on_filestore_osds = false 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: mon min osdmap epochs = 50 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: mon osd crush smoke test = False 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: mon osdmap full prune interval = 2 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: mon osdmap full prune min = 15 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: mon osdmap full prune txsize = 2 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: mon scrub interval = 300 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: mon warn on pool no app = False 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: paxos service trim min = 10 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout:[client] 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: rgw cache enabled = true 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: rgw enable ops log = true 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: rgw enable usage log = true 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: debug monc = 20 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: debug ms = 1 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: debug objecter = 20 2026-02-25T15:00:19.200 INFO:teuthology.orchestra.run.trial093.stdout: debug rados = 20 2026-02-25T15:00:19.201 INFO:teuthology.orchestra.run.trial093.stdout: rados replica read policy = balance 2026-02-25T15:00:19.201 INFO:teuthology.orchestra.run.trial093.stdout:[mon.a] 2026-02-25T15:00:19.201 INFO:teuthology.orchestra.run.trial093.stdout:[mon.b] 2026-02-25T15:00:19.201 INFO:teuthology.orchestra.run.trial093.stdout:[mon.c] 2026-02-25T15:00:19.202 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-25T15:00:19.202 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-25T15:00:19.275 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-25T15:00:19.276 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:19.276 DEBUG:teuthology.orchestra.run.trial090:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-25T15:00:19.292 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:19.292 DEBUG:teuthology.orchestra.run.trial090:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-25T15:00:19.348 INFO:tasks.ceph:Sending monmap to node ubuntu@trial090.front.sepia.ceph.com 2026-02-25T15:00:19.348 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:19.348 DEBUG:teuthology.orchestra.run.trial090:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-25T15:00:19.349 DEBUG:teuthology.orchestra.run.trial090:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T15:00:19.421 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:19.421 DEBUG:teuthology.orchestra.run.trial090:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:19.477 INFO:tasks.ceph:Sending monmap to node ubuntu@trial093.front.sepia.ceph.com 2026-02-25T15:00:19.477 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T15:00:19.477 DEBUG:teuthology.orchestra.run.trial093:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-25T15:00:19.477 DEBUG:teuthology.orchestra.run.trial093:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T15:00:19.511 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T15:00:19.511 DEBUG:teuthology.orchestra.run.trial093:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:19.567 INFO:tasks.ceph:Sending monmap to node ubuntu@trial094.front.sepia.ceph.com 2026-02-25T15:00:19.567 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T15:00:19.567 DEBUG:teuthology.orchestra.run.trial094:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-25T15:00:19.567 DEBUG:teuthology.orchestra.run.trial094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T15:00:19.600 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T15:00:19.601 DEBUG:teuthology.orchestra.run.trial094:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:19.656 INFO:tasks.ceph:Sending monmap to node ubuntu@trial189.front.sepia.ceph.com 2026-02-25T15:00:19.656 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:19.656 DEBUG:teuthology.orchestra.run.trial189:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-25T15:00:19.657 DEBUG:teuthology.orchestra.run.trial189:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T15:00:19.690 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:19.691 DEBUG:teuthology.orchestra.run.trial189:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:19.747 INFO:tasks.ceph:Setting up mon nodes... 2026-02-25T15:00:19.747 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-25T15:00:19.747 DEBUG:teuthology.orchestra.run.trial090:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2026-02-25T15:00:19.790 INFO:teuthology.orchestra.run.trial090.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2026-02-25T15:00:19.793 DEBUG:teuthology.orchestra.run.trial189:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-25T15:00:19.837 INFO:teuthology.orchestra.run.trial189.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-25T15:00:19.840 INFO:tasks.ceph:Setting up mds nodes... 2026-02-25T15:00:19.840 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-25T15:00:19.841 DEBUG:teuthology.orchestra.run.trial189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-25T15:00:19.915 INFO:teuthology.orchestra.run.trial189.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-25T15:00:19.926 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-25T15:00:19.927 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial090.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial093.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial094.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial189.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2026-02-25T15:00:19.927 DEBUG:teuthology.orchestra.run.trial090:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-25T15:00:19.952 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:19.952 INFO:tasks.ceph:role: osd.0 2026-02-25T15:00:19.953 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial090.front.sepia.ceph.com 2026-02-25T15:00:19.953 DEBUG:teuthology.orchestra.run.trial090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-25T15:00:20.016 INFO:teuthology.orchestra.run.trial090.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:20.016 INFO:teuthology.orchestra.run.trial090.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:20.016 INFO:teuthology.orchestra.run.trial090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:20.016 INFO:teuthology.orchestra.run.trial090.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:20.017 INFO:teuthology.orchestra.run.trial090.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:20.017 INFO:teuthology.orchestra.run.trial090.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:20.017 INFO:teuthology.orchestra.run.trial090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:20.017 INFO:teuthology.orchestra.run.trial090.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:20.017 INFO:teuthology.orchestra.run.trial090.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:20.017 INFO:teuthology.orchestra.run.trial090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:20.588 INFO:teuthology.orchestra.run.trial090.stdout:Discarding blocks...Done. 2026-02-25T15:00:20.590 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial090.front.sepia.ceph.com -o noatime 2026-02-25T15:00:20.590 DEBUG:teuthology.orchestra.run.trial090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-25T15:00:20.659 DEBUG:teuthology.orchestra.run.trial090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-25T15:00:20.726 DEBUG:teuthology.orchestra.run.trial090:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2026-02-25T15:00:20.790 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:20.790 INFO:tasks.ceph:role: osd.4 2026-02-25T15:00:20.790 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial090.front.sepia.ceph.com 2026-02-25T15:00:20.790 DEBUG:teuthology.orchestra.run.trial090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-25T15:00:20.854 INFO:teuthology.orchestra.run.trial090.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:20.855 INFO:teuthology.orchestra.run.trial090.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:20.855 INFO:teuthology.orchestra.run.trial090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:20.855 INFO:teuthology.orchestra.run.trial090.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:20.855 INFO:teuthology.orchestra.run.trial090.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:20.855 INFO:teuthology.orchestra.run.trial090.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:20.855 INFO:teuthology.orchestra.run.trial090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:20.855 INFO:teuthology.orchestra.run.trial090.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:20.855 INFO:teuthology.orchestra.run.trial090.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:20.855 INFO:teuthology.orchestra.run.trial090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:21.414 INFO:teuthology.orchestra.run.trial090.stdout:Discarding blocks...Done. 2026-02-25T15:00:21.416 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial090.front.sepia.ceph.com -o noatime 2026-02-25T15:00:21.416 DEBUG:teuthology.orchestra.run.trial090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2026-02-25T15:00:21.483 DEBUG:teuthology.orchestra.run.trial090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2026-02-25T15:00:21.551 DEBUG:teuthology.orchestra.run.trial090:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2026-02-25T15:00:21.615 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:21.615 INFO:tasks.ceph:role: osd.8 2026-02-25T15:00:21.615 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial090.front.sepia.ceph.com 2026-02-25T15:00:21.616 DEBUG:teuthology.orchestra.run.trial090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-25T15:00:21.679 INFO:teuthology.orchestra.run.trial090.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:21.679 INFO:teuthology.orchestra.run.trial090.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:21.679 INFO:teuthology.orchestra.run.trial090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:21.679 INFO:teuthology.orchestra.run.trial090.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:21.679 INFO:teuthology.orchestra.run.trial090.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:21.680 INFO:teuthology.orchestra.run.trial090.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:21.680 INFO:teuthology.orchestra.run.trial090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:21.680 INFO:teuthology.orchestra.run.trial090.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:21.680 INFO:teuthology.orchestra.run.trial090.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:21.680 INFO:teuthology.orchestra.run.trial090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:22.265 INFO:teuthology.orchestra.run.trial090.stdout:Discarding blocks...Done. 2026-02-25T15:00:22.266 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial090.front.sepia.ceph.com -o noatime 2026-02-25T15:00:22.267 DEBUG:teuthology.orchestra.run.trial090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-8 2026-02-25T15:00:22.333 DEBUG:teuthology.orchestra.run.trial090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2026-02-25T15:00:22.399 DEBUG:teuthology.orchestra.run.trial090:> sudo mkdir -p /var/lib/ceph/osd/ceph-12 2026-02-25T15:00:22.463 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:22.463 INFO:tasks.ceph:role: osd.12 2026-02-25T15:00:22.463 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial090.front.sepia.ceph.com 2026-02-25T15:00:22.463 DEBUG:teuthology.orchestra.run.trial090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-25T15:00:22.528 INFO:teuthology.orchestra.run.trial090.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:22.528 INFO:teuthology.orchestra.run.trial090.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:22.528 INFO:teuthology.orchestra.run.trial090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:22.528 INFO:teuthology.orchestra.run.trial090.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:22.529 INFO:teuthology.orchestra.run.trial090.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:22.529 INFO:teuthology.orchestra.run.trial090.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:22.529 INFO:teuthology.orchestra.run.trial090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:22.529 INFO:teuthology.orchestra.run.trial090.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:22.529 INFO:teuthology.orchestra.run.trial090.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:22.529 INFO:teuthology.orchestra.run.trial090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:23.083 INFO:teuthology.orchestra.run.trial090.stdout:Discarding blocks...Done. 2026-02-25T15:00:23.085 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial090.front.sepia.ceph.com -o noatime 2026-02-25T15:00:23.085 DEBUG:teuthology.orchestra.run.trial090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-12 2026-02-25T15:00:23.169 DEBUG:teuthology.orchestra.run.trial090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-12 2026-02-25T15:00:23.236 DEBUG:teuthology.orchestra.run.trial090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:23.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:23.314+0000 7fb2f000e900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-25T15:00:23.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:23.314+0000 7fb2f000e900 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-25T15:00:23.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:23.314+0000 7fb2f000e900 -1 bdev(0x563ae3b8f800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:23.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:23.314+0000 7fb2f000e900 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-25T15:00:23.787 DEBUG:teuthology.orchestra.run.trial090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-25T15:00:23.852 DEBUG:teuthology.orchestra.run.trial090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:23.930 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:23.929+0000 7efdfa472900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2026-02-25T15:00:23.931 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:23.929+0000 7efdfa472900 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2026-02-25T15:00:23.931 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:23.929+0000 7efdfa472900 -1 bdev(0x564f73f57800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:23.931 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:23.929+0000 7efdfa472900 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2026-02-25T15:00:24.426 DEBUG:teuthology.orchestra.run.trial090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2026-02-25T15:00:24.490 DEBUG:teuthology.orchestra.run.trial090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:24.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:24.567+0000 7f3779518900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2026-02-25T15:00:24.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:24.567+0000 7f3779518900 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2026-02-25T15:00:24.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:24.567+0000 7f3779518900 -1 bdev(0x55dc59563800 /var/lib/ceph/osd/ceph-8/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:24.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:24.567+0000 7f3779518900 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2026-02-25T15:00:25.070 DEBUG:teuthology.orchestra.run.trial090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2026-02-25T15:00:25.136 DEBUG:teuthology.orchestra.run.trial090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 12 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:25.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:25.213+0000 7feafbc0f900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-12/keyring: can't open /var/lib/ceph/osd/ceph-12/keyring: (2) No such file or directory 2026-02-25T15:00:25.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:25.213+0000 7feafbc0f900 -1 created new key in keyring /var/lib/ceph/osd/ceph-12/keyring 2026-02-25T15:00:25.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:25.213+0000 7feafbc0f900 -1 bdev(0x558035d49800 /var/lib/ceph/osd/ceph-12/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:25.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:25.213+0000 7feafbc0f900 -1 bluestore(/var/lib/ceph/osd/ceph-12) _read_fsid unparsable uuid 2026-02-25T15:00:25.709 DEBUG:teuthology.orchestra.run.trial090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-12 2026-02-25T15:00:25.773 DEBUG:teuthology.orchestra.run.trial093:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-25T15:00:25.799 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:25.799 INFO:tasks.ceph:role: osd.1 2026-02-25T15:00:25.799 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial093.front.sepia.ceph.com 2026-02-25T15:00:25.799 DEBUG:teuthology.orchestra.run.trial093:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-25T15:00:25.864 INFO:teuthology.orchestra.run.trial093.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:25.864 INFO:teuthology.orchestra.run.trial093.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:25.864 INFO:teuthology.orchestra.run.trial093.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:25.864 INFO:teuthology.orchestra.run.trial093.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:25.864 INFO:teuthology.orchestra.run.trial093.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:25.864 INFO:teuthology.orchestra.run.trial093.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:25.865 INFO:teuthology.orchestra.run.trial093.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:25.865 INFO:teuthology.orchestra.run.trial093.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:25.865 INFO:teuthology.orchestra.run.trial093.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:25.865 INFO:teuthology.orchestra.run.trial093.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:26.433 INFO:teuthology.orchestra.run.trial093.stdout:Discarding blocks...Done. 2026-02-25T15:00:26.435 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial093.front.sepia.ceph.com -o noatime 2026-02-25T15:00:26.435 DEBUG:teuthology.orchestra.run.trial093:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2026-02-25T15:00:26.503 DEBUG:teuthology.orchestra.run.trial093:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-25T15:00:26.569 DEBUG:teuthology.orchestra.run.trial093:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2026-02-25T15:00:26.633 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:26.633 INFO:tasks.ceph:role: osd.5 2026-02-25T15:00:26.633 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial093.front.sepia.ceph.com 2026-02-25T15:00:26.633 DEBUG:teuthology.orchestra.run.trial093:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-25T15:00:26.698 INFO:teuthology.orchestra.run.trial093.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:26.698 INFO:teuthology.orchestra.run.trial093.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:26.698 INFO:teuthology.orchestra.run.trial093.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:26.698 INFO:teuthology.orchestra.run.trial093.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:26.698 INFO:teuthology.orchestra.run.trial093.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:26.698 INFO:teuthology.orchestra.run.trial093.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:26.698 INFO:teuthology.orchestra.run.trial093.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:26.698 INFO:teuthology.orchestra.run.trial093.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:26.698 INFO:teuthology.orchestra.run.trial093.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:26.698 INFO:teuthology.orchestra.run.trial093.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:27.255 INFO:teuthology.orchestra.run.trial093.stdout:Discarding blocks...Done. 2026-02-25T15:00:27.257 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial093.front.sepia.ceph.com -o noatime 2026-02-25T15:00:27.257 DEBUG:teuthology.orchestra.run.trial093:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2026-02-25T15:00:27.324 DEBUG:teuthology.orchestra.run.trial093:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2026-02-25T15:00:27.391 DEBUG:teuthology.orchestra.run.trial093:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2026-02-25T15:00:27.455 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:27.456 INFO:tasks.ceph:role: osd.9 2026-02-25T15:00:27.456 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial093.front.sepia.ceph.com 2026-02-25T15:00:27.456 DEBUG:teuthology.orchestra.run.trial093:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-25T15:00:27.520 INFO:teuthology.orchestra.run.trial093.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:27.521 INFO:teuthology.orchestra.run.trial093.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:27.521 INFO:teuthology.orchestra.run.trial093.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:27.521 INFO:teuthology.orchestra.run.trial093.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:27.521 INFO:teuthology.orchestra.run.trial093.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:27.521 INFO:teuthology.orchestra.run.trial093.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:27.521 INFO:teuthology.orchestra.run.trial093.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:27.521 INFO:teuthology.orchestra.run.trial093.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:27.521 INFO:teuthology.orchestra.run.trial093.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:27.521 INFO:teuthology.orchestra.run.trial093.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:28.076 INFO:teuthology.orchestra.run.trial093.stdout:Discarding blocks...Done. 2026-02-25T15:00:28.078 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial093.front.sepia.ceph.com -o noatime 2026-02-25T15:00:28.079 DEBUG:teuthology.orchestra.run.trial093:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-9 2026-02-25T15:00:28.147 DEBUG:teuthology.orchestra.run.trial093:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2026-02-25T15:00:28.212 DEBUG:teuthology.orchestra.run.trial093:> sudo mkdir -p /var/lib/ceph/osd/ceph-13 2026-02-25T15:00:28.276 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:28.276 INFO:tasks.ceph:role: osd.13 2026-02-25T15:00:28.277 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial093.front.sepia.ceph.com 2026-02-25T15:00:28.277 DEBUG:teuthology.orchestra.run.trial093:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-25T15:00:28.341 INFO:teuthology.orchestra.run.trial093.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:28.341 INFO:teuthology.orchestra.run.trial093.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:28.341 INFO:teuthology.orchestra.run.trial093.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:28.341 INFO:teuthology.orchestra.run.trial093.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:28.342 INFO:teuthology.orchestra.run.trial093.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:28.342 INFO:teuthology.orchestra.run.trial093.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:28.342 INFO:teuthology.orchestra.run.trial093.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:28.342 INFO:teuthology.orchestra.run.trial093.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:28.342 INFO:teuthology.orchestra.run.trial093.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:28.342 INFO:teuthology.orchestra.run.trial093.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:28.897 INFO:teuthology.orchestra.run.trial093.stdout:Discarding blocks...Done. 2026-02-25T15:00:28.899 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial093.front.sepia.ceph.com -o noatime 2026-02-25T15:00:28.899 DEBUG:teuthology.orchestra.run.trial093:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-13 2026-02-25T15:00:28.976 DEBUG:teuthology.orchestra.run.trial093:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-13 2026-02-25T15:00:29.044 DEBUG:teuthology.orchestra.run.trial093:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:29.123 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:29.120+0000 7f071540c900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-25T15:00:29.124 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:29.120+0000 7f071540c900 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-25T15:00:29.124 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:29.121+0000 7f071540c900 -1 bdev(0x560e16ee1800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:29.125 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:29.121+0000 7f071540c900 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-25T15:00:29.621 DEBUG:teuthology.orchestra.run.trial093:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-25T15:00:29.685 DEBUG:teuthology.orchestra.run.trial093:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:29.763 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:29.759+0000 7f0b9d010900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2026-02-25T15:00:29.763 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:29.759+0000 7f0b9d010900 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2026-02-25T15:00:29.763 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:29.760+0000 7f0b9d010900 -1 bdev(0x55834df6f800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:29.763 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:29.760+0000 7f0b9d010900 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2026-02-25T15:00:30.253 DEBUG:teuthology.orchestra.run.trial093:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2026-02-25T15:00:30.317 DEBUG:teuthology.orchestra.run.trial093:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:30.397 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:30.393+0000 7f624fed2900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2026-02-25T15:00:30.397 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:30.393+0000 7f624fed2900 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2026-02-25T15:00:30.397 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:30.393+0000 7f624fed2900 -1 bdev(0x555a27549800 /var/lib/ceph/osd/ceph-9/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:30.397 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:30.393+0000 7f624fed2900 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2026-02-25T15:00:30.892 DEBUG:teuthology.orchestra.run.trial093:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2026-02-25T15:00:30.958 DEBUG:teuthology.orchestra.run.trial093:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 13 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:31.035 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:31.032+0000 7f2d4ed7f900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-13/keyring: can't open /var/lib/ceph/osd/ceph-13/keyring: (2) No such file or directory 2026-02-25T15:00:31.036 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:31.032+0000 7f2d4ed7f900 -1 created new key in keyring /var/lib/ceph/osd/ceph-13/keyring 2026-02-25T15:00:31.036 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:31.032+0000 7f2d4ed7f900 -1 bdev(0x556eb314f800 /var/lib/ceph/osd/ceph-13/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:31.036 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-25T15:00:31.032+0000 7f2d4ed7f900 -1 bluestore(/var/lib/ceph/osd/ceph-13) _read_fsid unparsable uuid 2026-02-25T15:00:31.546 DEBUG:teuthology.orchestra.run.trial093:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-13 2026-02-25T15:00:31.611 DEBUG:teuthology.orchestra.run.trial094:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-25T15:00:31.637 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:31.637 INFO:tasks.ceph:role: osd.2 2026-02-25T15:00:31.637 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial094.front.sepia.ceph.com 2026-02-25T15:00:31.638 DEBUG:teuthology.orchestra.run.trial094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-25T15:00:31.702 INFO:teuthology.orchestra.run.trial094.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:31.703 INFO:teuthology.orchestra.run.trial094.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:31.703 INFO:teuthology.orchestra.run.trial094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:31.703 INFO:teuthology.orchestra.run.trial094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:31.703 INFO:teuthology.orchestra.run.trial094.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:31.703 INFO:teuthology.orchestra.run.trial094.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:31.703 INFO:teuthology.orchestra.run.trial094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:31.703 INFO:teuthology.orchestra.run.trial094.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:31.703 INFO:teuthology.orchestra.run.trial094.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:31.703 INFO:teuthology.orchestra.run.trial094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:32.274 INFO:teuthology.orchestra.run.trial094.stdout:Discarding blocks...Done. 2026-02-25T15:00:32.276 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial094.front.sepia.ceph.com -o noatime 2026-02-25T15:00:32.276 DEBUG:teuthology.orchestra.run.trial094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2026-02-25T15:00:32.344 DEBUG:teuthology.orchestra.run.trial094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-25T15:00:32.411 DEBUG:teuthology.orchestra.run.trial094:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2026-02-25T15:00:32.475 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:32.476 INFO:tasks.ceph:role: osd.6 2026-02-25T15:00:32.476 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial094.front.sepia.ceph.com 2026-02-25T15:00:32.476 DEBUG:teuthology.orchestra.run.trial094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-25T15:00:32.540 INFO:teuthology.orchestra.run.trial094.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:32.540 INFO:teuthology.orchestra.run.trial094.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:32.540 INFO:teuthology.orchestra.run.trial094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:32.540 INFO:teuthology.orchestra.run.trial094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:32.541 INFO:teuthology.orchestra.run.trial094.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:32.541 INFO:teuthology.orchestra.run.trial094.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:32.541 INFO:teuthology.orchestra.run.trial094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:32.541 INFO:teuthology.orchestra.run.trial094.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:32.541 INFO:teuthology.orchestra.run.trial094.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:32.541 INFO:teuthology.orchestra.run.trial094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:33.114 INFO:teuthology.orchestra.run.trial094.stdout:Discarding blocks...Done. 2026-02-25T15:00:33.116 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial094.front.sepia.ceph.com -o noatime 2026-02-25T15:00:33.116 DEBUG:teuthology.orchestra.run.trial094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2026-02-25T15:00:33.184 DEBUG:teuthology.orchestra.run.trial094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2026-02-25T15:00:33.251 DEBUG:teuthology.orchestra.run.trial094:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2026-02-25T15:00:33.314 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:33.314 INFO:tasks.ceph:role: osd.10 2026-02-25T15:00:33.314 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial094.front.sepia.ceph.com 2026-02-25T15:00:33.315 DEBUG:teuthology.orchestra.run.trial094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-25T15:00:33.379 INFO:teuthology.orchestra.run.trial094.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:33.379 INFO:teuthology.orchestra.run.trial094.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:33.379 INFO:teuthology.orchestra.run.trial094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:33.379 INFO:teuthology.orchestra.run.trial094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:33.379 INFO:teuthology.orchestra.run.trial094.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:33.380 INFO:teuthology.orchestra.run.trial094.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:33.380 INFO:teuthology.orchestra.run.trial094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:33.380 INFO:teuthology.orchestra.run.trial094.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:33.380 INFO:teuthology.orchestra.run.trial094.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:33.380 INFO:teuthology.orchestra.run.trial094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:33.967 INFO:teuthology.orchestra.run.trial094.stdout:Discarding blocks...Done. 2026-02-25T15:00:33.969 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial094.front.sepia.ceph.com -o noatime 2026-02-25T15:00:33.969 DEBUG:teuthology.orchestra.run.trial094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2026-02-25T15:00:34.037 DEBUG:teuthology.orchestra.run.trial094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2026-02-25T15:00:34.104 DEBUG:teuthology.orchestra.run.trial094:> sudo mkdir -p /var/lib/ceph/osd/ceph-14 2026-02-25T15:00:34.168 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:34.168 INFO:tasks.ceph:role: osd.14 2026-02-25T15:00:34.168 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial094.front.sepia.ceph.com 2026-02-25T15:00:34.168 DEBUG:teuthology.orchestra.run.trial094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-25T15:00:34.233 INFO:teuthology.orchestra.run.trial094.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:34.233 INFO:teuthology.orchestra.run.trial094.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:34.233 INFO:teuthology.orchestra.run.trial094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:34.233 INFO:teuthology.orchestra.run.trial094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:34.233 INFO:teuthology.orchestra.run.trial094.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:34.233 INFO:teuthology.orchestra.run.trial094.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:34.233 INFO:teuthology.orchestra.run.trial094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:34.233 INFO:teuthology.orchestra.run.trial094.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:34.234 INFO:teuthology.orchestra.run.trial094.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:34.234 INFO:teuthology.orchestra.run.trial094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:34.788 INFO:teuthology.orchestra.run.trial094.stdout:Discarding blocks...Done. 2026-02-25T15:00:34.790 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial094.front.sepia.ceph.com -o noatime 2026-02-25T15:00:34.791 DEBUG:teuthology.orchestra.run.trial094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-14 2026-02-25T15:00:34.869 DEBUG:teuthology.orchestra.run.trial094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-14 2026-02-25T15:00:34.936 DEBUG:teuthology.orchestra.run.trial094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:35.016 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:35.014+0000 7f0506e0d900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2026-02-25T15:00:35.017 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:35.015+0000 7f0506e0d900 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2026-02-25T15:00:35.017 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:35.015+0000 7f0506e0d900 -1 bdev(0x563f744e5800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:35.017 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:35.015+0000 7f0506e0d900 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2026-02-25T15:00:35.489 DEBUG:teuthology.orchestra.run.trial094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-25T15:00:35.554 DEBUG:teuthology.orchestra.run.trial094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:35.631 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:35.629+0000 7fd369822900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2026-02-25T15:00:35.631 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:35.629+0000 7fd369822900 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2026-02-25T15:00:35.632 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:35.630+0000 7fd369822900 -1 bdev(0x56408488d800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:35.632 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:35.630+0000 7fd369822900 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2026-02-25T15:00:36.109 DEBUG:teuthology.orchestra.run.trial094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2026-02-25T15:00:36.174 DEBUG:teuthology.orchestra.run.trial094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:36.253 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:36.251+0000 7fc2d5c63900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2026-02-25T15:00:36.253 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:36.251+0000 7fc2d5c63900 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2026-02-25T15:00:36.253 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:36.251+0000 7fc2d5c63900 -1 bdev(0x55d2336a1800 /var/lib/ceph/osd/ceph-10/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:36.254 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:36.251+0000 7fc2d5c63900 -1 bluestore(/var/lib/ceph/osd/ceph-10) _read_fsid unparsable uuid 2026-02-25T15:00:36.757 DEBUG:teuthology.orchestra.run.trial094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2026-02-25T15:00:36.822 DEBUG:teuthology.orchestra.run.trial094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 14 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:36.901 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:36.899+0000 7f53d2215900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-14/keyring: can't open /var/lib/ceph/osd/ceph-14/keyring: (2) No such file or directory 2026-02-25T15:00:36.901 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:36.900+0000 7f53d2215900 -1 created new key in keyring /var/lib/ceph/osd/ceph-14/keyring 2026-02-25T15:00:36.901 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:36.900+0000 7f53d2215900 -1 bdev(0x55b1ec917800 /var/lib/ceph/osd/ceph-14/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:36.902 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T15:00:36.900+0000 7f53d2215900 -1 bluestore(/var/lib/ceph/osd/ceph-14) _read_fsid unparsable uuid 2026-02-25T15:00:37.389 DEBUG:teuthology.orchestra.run.trial094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-14 2026-02-25T15:00:37.455 DEBUG:teuthology.orchestra.run.trial189:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-25T15:00:37.481 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:37.481 INFO:tasks.ceph:role: osd.3 2026-02-25T15:00:37.481 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial189.front.sepia.ceph.com 2026-02-25T15:00:37.481 DEBUG:teuthology.orchestra.run.trial189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-25T15:00:37.546 INFO:teuthology.orchestra.run.trial189.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:37.547 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:37.547 INFO:teuthology.orchestra.run.trial189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:37.547 INFO:teuthology.orchestra.run.trial189.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:37.547 INFO:teuthology.orchestra.run.trial189.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:37.547 INFO:teuthology.orchestra.run.trial189.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:37.547 INFO:teuthology.orchestra.run.trial189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:37.547 INFO:teuthology.orchestra.run.trial189.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:37.547 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:37.547 INFO:teuthology.orchestra.run.trial189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:38.114 INFO:teuthology.orchestra.run.trial189.stdout:Discarding blocks...Done. 2026-02-25T15:00:38.116 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial189.front.sepia.ceph.com -o noatime 2026-02-25T15:00:38.116 DEBUG:teuthology.orchestra.run.trial189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2026-02-25T15:00:38.184 DEBUG:teuthology.orchestra.run.trial189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-25T15:00:38.250 DEBUG:teuthology.orchestra.run.trial189:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2026-02-25T15:00:38.315 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:38.316 INFO:tasks.ceph:role: osd.7 2026-02-25T15:00:38.316 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial189.front.sepia.ceph.com 2026-02-25T15:00:38.316 DEBUG:teuthology.orchestra.run.trial189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-25T15:00:38.381 INFO:teuthology.orchestra.run.trial189.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:38.381 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:38.381 INFO:teuthology.orchestra.run.trial189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:38.381 INFO:teuthology.orchestra.run.trial189.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:38.381 INFO:teuthology.orchestra.run.trial189.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:38.381 INFO:teuthology.orchestra.run.trial189.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:38.381 INFO:teuthology.orchestra.run.trial189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:38.382 INFO:teuthology.orchestra.run.trial189.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:38.382 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:38.382 INFO:teuthology.orchestra.run.trial189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:38.936 INFO:teuthology.orchestra.run.trial189.stdout:Discarding blocks...Done. 2026-02-25T15:00:38.938 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial189.front.sepia.ceph.com -o noatime 2026-02-25T15:00:38.938 DEBUG:teuthology.orchestra.run.trial189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-7 2026-02-25T15:00:39.006 DEBUG:teuthology.orchestra.run.trial189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2026-02-25T15:00:39.072 DEBUG:teuthology.orchestra.run.trial189:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2026-02-25T15:00:39.136 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:39.136 INFO:tasks.ceph:role: osd.11 2026-02-25T15:00:39.136 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial189.front.sepia.ceph.com 2026-02-25T15:00:39.137 DEBUG:teuthology.orchestra.run.trial189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-25T15:00:39.200 INFO:teuthology.orchestra.run.trial189.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:39.201 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:39.201 INFO:teuthology.orchestra.run.trial189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:39.201 INFO:teuthology.orchestra.run.trial189.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:39.201 INFO:teuthology.orchestra.run.trial189.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:39.201 INFO:teuthology.orchestra.run.trial189.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:39.201 INFO:teuthology.orchestra.run.trial189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:39.201 INFO:teuthology.orchestra.run.trial189.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:39.201 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:39.201 INFO:teuthology.orchestra.run.trial189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:39.757 INFO:teuthology.orchestra.run.trial189.stdout:Discarding blocks...Done. 2026-02-25T15:00:39.759 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial189.front.sepia.ceph.com -o noatime 2026-02-25T15:00:39.759 DEBUG:teuthology.orchestra.run.trial189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-11 2026-02-25T15:00:39.827 DEBUG:teuthology.orchestra.run.trial189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2026-02-25T15:00:39.893 DEBUG:teuthology.orchestra.run.trial189:> sudo mkdir -p /var/lib/ceph/osd/ceph-15 2026-02-25T15:00:39.957 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-25T15:00:39.957 INFO:tasks.ceph:role: osd.15 2026-02-25T15:00:39.957 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial189.front.sepia.ceph.com 2026-02-25T15:00:39.958 DEBUG:teuthology.orchestra.run.trial189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-25T15:00:40.021 INFO:teuthology.orchestra.run.trial189.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T15:00:40.021 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T15:00:40.022 INFO:teuthology.orchestra.run.trial189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T15:00:40.022 INFO:teuthology.orchestra.run.trial189.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T15:00:40.022 INFO:teuthology.orchestra.run.trial189.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T15:00:40.022 INFO:teuthology.orchestra.run.trial189.stdout: = sunit=0 swidth=0 blks 2026-02-25T15:00:40.022 INFO:teuthology.orchestra.run.trial189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T15:00:40.022 INFO:teuthology.orchestra.run.trial189.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T15:00:40.022 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T15:00:40.022 INFO:teuthology.orchestra.run.trial189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T15:00:40.578 INFO:teuthology.orchestra.run.trial189.stdout:Discarding blocks...Done. 2026-02-25T15:00:40.580 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial189.front.sepia.ceph.com -o noatime 2026-02-25T15:00:40.580 DEBUG:teuthology.orchestra.run.trial189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-15 2026-02-25T15:00:40.657 DEBUG:teuthology.orchestra.run.trial189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-15 2026-02-25T15:00:40.725 DEBUG:teuthology.orchestra.run.trial189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:40.804 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:40.803+0000 7fbc5825d900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2026-02-25T15:00:40.805 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:40.803+0000 7fbc5825d900 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2026-02-25T15:00:40.805 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:40.804+0000 7fbc5825d900 -1 bdev(0x55687d28d800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:40.805 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:40.804+0000 7fbc5825d900 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2026-02-25T15:00:41.278 DEBUG:teuthology.orchestra.run.trial189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-25T15:00:41.344 DEBUG:teuthology.orchestra.run.trial189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:41.422 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:41.421+0000 7f00cb27d900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2026-02-25T15:00:41.423 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:41.421+0000 7f00cb27d900 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2026-02-25T15:00:41.423 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:41.421+0000 7f00cb27d900 -1 bdev(0x5569bf359800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:41.423 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:41.421+0000 7f00cb27d900 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2026-02-25T15:00:41.926 DEBUG:teuthology.orchestra.run.trial189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2026-02-25T15:00:41.991 DEBUG:teuthology.orchestra.run.trial189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:42.071 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:42.070+0000 7f6e8453e900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2026-02-25T15:00:42.072 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:42.070+0000 7f6e8453e900 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2026-02-25T15:00:42.072 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:42.070+0000 7f6e8453e900 -1 bdev(0x56344b629800 /var/lib/ceph/osd/ceph-11/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:42.072 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:42.071+0000 7f6e8453e900 -1 bluestore(/var/lib/ceph/osd/ceph-11) _read_fsid unparsable uuid 2026-02-25T15:00:42.552 DEBUG:teuthology.orchestra.run.trial189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2026-02-25T15:00:42.618 DEBUG:teuthology.orchestra.run.trial189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 15 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:42.696 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:42.695+0000 7f97ac505900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-15/keyring: can't open /var/lib/ceph/osd/ceph-15/keyring: (2) No such file or directory 2026-02-25T15:00:42.697 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:42.696+0000 7f97ac505900 -1 created new key in keyring /var/lib/ceph/osd/ceph-15/keyring 2026-02-25T15:00:42.697 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:42.696+0000 7f97ac505900 -1 bdev(0x55cadd4a3800 /var/lib/ceph/osd/ceph-15/block) open stat got: (1) Operation not permitted 2026-02-25T15:00:42.697 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:42.696+0000 7f97ac505900 -1 bluestore(/var/lib/ceph/osd/ceph-15) _read_fsid unparsable uuid 2026-02-25T15:00:43.186 DEBUG:teuthology.orchestra.run.trial189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-15 2026-02-25T15:00:43.251 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-25T15:00:43.251 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:43.251 DEBUG:teuthology.orchestra.run.trial090:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2026-02-25T15:00:43.276 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:43.276 DEBUG:teuthology.orchestra.run.trial090:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-25T15:00:43.339 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:43.339 DEBUG:teuthology.orchestra.run.trial090:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2026-02-25T15:00:43.401 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:43.401 DEBUG:teuthology.orchestra.run.trial090:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2026-02-25T15:00:43.466 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:43.466 DEBUG:teuthology.orchestra.run.trial090:> sudo dd if=/var/lib/ceph/osd/ceph-12/keyring of=/dev/stdout 2026-02-25T15:00:43.529 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T15:00:43.529 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-25T15:00:43.555 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T15:00:43.555 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2026-02-25T15:00:43.621 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T15:00:43.621 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2026-02-25T15:00:43.685 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T15:00:43.686 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/var/lib/ceph/osd/ceph-13/keyring of=/dev/stdout 2026-02-25T15:00:43.749 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T15:00:43.749 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-25T15:00:43.775 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T15:00:43.775 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2026-02-25T15:00:43.839 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T15:00:43.839 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2026-02-25T15:00:43.903 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T15:00:43.904 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/var/lib/ceph/osd/ceph-14/keyring of=/dev/stdout 2026-02-25T15:00:43.967 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:43.967 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-25T15:00:43.993 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:43.993 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-25T15:00:44.057 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:44.057 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2026-02-25T15:00:44.121 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:44.121 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2026-02-25T15:00:44.184 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:44.185 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-15/keyring of=/dev/stdout 2026-02-25T15:00:44.249 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:44.250 DEBUG:teuthology.orchestra.run.trial189:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-25T15:00:44.306 INFO:tasks.ceph:Adding keys to all mons... 2026-02-25T15:00:44.307 DEBUG:teuthology.orchestra.run.trial090:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-25T15:00:44.309 DEBUG:teuthology.orchestra.run.trial093:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-25T15:00:44.312 DEBUG:teuthology.orchestra.run.trial094:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-25T15:00:44.331 INFO:teuthology.orchestra.run.trial090.stdout:[mgr.y] 2026-02-25T15:00:44.331 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCDDp9p2N0eLxAA3xJMFu/2Tp0I92UB5m6iBQ== 2026-02-25T15:00:44.331 INFO:teuthology.orchestra.run.trial090.stdout:[osd.0] 2026-02-25T15:00:44.331 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCHDp9p2qfGEhAA9xuhO9hotxkOeq2rK5aqlQ== 2026-02-25T15:00:44.331 INFO:teuthology.orchestra.run.trial090.stdout:[osd.4] 2026-02-25T15:00:44.332 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCHDp9p6StrNxAA/kgj8t5+iFcbyvx6yEZWhw== 2026-02-25T15:00:44.332 INFO:teuthology.orchestra.run.trial090.stdout:[osd.8] 2026-02-25T15:00:44.332 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCIDp9pUi/YIRAA72YP7AM2oW7n7DjnDkQCtQ== 2026-02-25T15:00:44.332 INFO:teuthology.orchestra.run.trial090.stdout:[osd.12] 2026-02-25T15:00:44.332 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCJDp9piB7DDBAABibxZ6vnVNz5HLPEH0rYzQ== 2026-02-25T15:00:44.332 INFO:teuthology.orchestra.run.trial090.stdout:[osd.1] 2026-02-25T15:00:44.332 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCNDp9p3hFKBxAA+WxSrSbsFmOoBVIWQWr4/Q== 2026-02-25T15:00:44.332 INFO:teuthology.orchestra.run.trial090.stdout:[osd.5] 2026-02-25T15:00:44.332 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCNDp9pw81gLRAAIsVcuE3wI4IFAXK2Yp42cQ== 2026-02-25T15:00:44.332 INFO:teuthology.orchestra.run.trial090.stdout:[osd.9] 2026-02-25T15:00:44.332 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCODp9pGpWNFxAAX3ilnxzAqGRV0ZSzw7mZVA== 2026-02-25T15:00:44.332 INFO:teuthology.orchestra.run.trial090.stdout:[osd.13] 2026-02-25T15:00:44.333 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCPDp9pRAoFAhAA0IkHABKTkXB9rYDDJp7CWA== 2026-02-25T15:00:44.333 INFO:teuthology.orchestra.run.trial090.stdout:[osd.2] 2026-02-25T15:00:44.333 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCTDp9p7gL0ABAALNonEfqr3o+VLJyw39SDwg== 2026-02-25T15:00:44.333 INFO:teuthology.orchestra.run.trial090.stdout:[osd.6] 2026-02-25T15:00:44.333 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCTDp9p5NyZJRAAVkTATRj4wmeXD1dS0Hx2nQ== 2026-02-25T15:00:44.333 INFO:teuthology.orchestra.run.trial090.stdout:[osd.10] 2026-02-25T15:00:44.333 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCUDp9pkboPDxAARS1PBlAsdqW5nnhg7EucNw== 2026-02-25T15:00:44.333 INFO:teuthology.orchestra.run.trial090.stdout:[osd.14] 2026-02-25T15:00:44.333 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCUDp9pe56zNRAA9FbMbMUOW4inYbhq8I63tA== 2026-02-25T15:00:44.333 INFO:teuthology.orchestra.run.trial090.stdout:[mgr.x] 2026-02-25T15:00:44.333 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCDDp9pXBX1MRAA8L7B4QTwmHasuEDpHX6uKw== 2026-02-25T15:00:44.333 INFO:teuthology.orchestra.run.trial090.stdout:[osd.3] 2026-02-25T15:00:44.334 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCYDp9ppJL6LxAATUMo1puxGvIvx7TFfY3dGg== 2026-02-25T15:00:44.334 INFO:teuthology.orchestra.run.trial090.stdout:[osd.7] 2026-02-25T15:00:44.334 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCZDp9pMUExGRAATGPGt5Ns8QjCij/6gh6d8g== 2026-02-25T15:00:44.334 INFO:teuthology.orchestra.run.trial090.stdout:[osd.11] 2026-02-25T15:00:44.334 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCaDp9pcDRIBBAAoa/68vOZoTPqYjAIH+Jwqg== 2026-02-25T15:00:44.334 INFO:teuthology.orchestra.run.trial090.stdout:[osd.15] 2026-02-25T15:00:44.334 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCaDp9prxeMKRAAprXwUKp/eFEq3WxZhWSO2Q== 2026-02-25T15:00:44.334 INFO:teuthology.orchestra.run.trial090.stdout:[client.0] 2026-02-25T15:00:44.334 INFO:teuthology.orchestra.run.trial090.stdout: key = AQCDDp9pbQmUNhAAGoUNoVUS8TJ7XSbHbD7w9w== 2026-02-25T15:00:44.336 INFO:teuthology.orchestra.run.trial093.stdout:[mgr.y] 2026-02-25T15:00:44.336 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCDDp9p2N0eLxAA3xJMFu/2Tp0I92UB5m6iBQ== 2026-02-25T15:00:44.336 INFO:teuthology.orchestra.run.trial093.stdout:[osd.0] 2026-02-25T15:00:44.336 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCHDp9p2qfGEhAA9xuhO9hotxkOeq2rK5aqlQ== 2026-02-25T15:00:44.336 INFO:teuthology.orchestra.run.trial093.stdout:[osd.4] 2026-02-25T15:00:44.336 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCHDp9p6StrNxAA/kgj8t5+iFcbyvx6yEZWhw== 2026-02-25T15:00:44.337 INFO:teuthology.orchestra.run.trial093.stdout:[osd.8] 2026-02-25T15:00:44.337 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCIDp9pUi/YIRAA72YP7AM2oW7n7DjnDkQCtQ== 2026-02-25T15:00:44.337 INFO:teuthology.orchestra.run.trial093.stdout:[osd.12] 2026-02-25T15:00:44.337 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCJDp9piB7DDBAABibxZ6vnVNz5HLPEH0rYzQ== 2026-02-25T15:00:44.337 INFO:teuthology.orchestra.run.trial093.stdout:[osd.1] 2026-02-25T15:00:44.337 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCNDp9p3hFKBxAA+WxSrSbsFmOoBVIWQWr4/Q== 2026-02-25T15:00:44.337 INFO:teuthology.orchestra.run.trial093.stdout:[osd.5] 2026-02-25T15:00:44.337 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCNDp9pw81gLRAAIsVcuE3wI4IFAXK2Yp42cQ== 2026-02-25T15:00:44.337 INFO:teuthology.orchestra.run.trial093.stdout:[osd.9] 2026-02-25T15:00:44.337 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCODp9pGpWNFxAAX3ilnxzAqGRV0ZSzw7mZVA== 2026-02-25T15:00:44.337 INFO:teuthology.orchestra.run.trial093.stdout:[osd.13] 2026-02-25T15:00:44.338 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCPDp9pRAoFAhAA0IkHABKTkXB9rYDDJp7CWA== 2026-02-25T15:00:44.338 INFO:teuthology.orchestra.run.trial093.stdout:[osd.2] 2026-02-25T15:00:44.338 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCTDp9p7gL0ABAALNonEfqr3o+VLJyw39SDwg== 2026-02-25T15:00:44.338 INFO:teuthology.orchestra.run.trial093.stdout:[osd.6] 2026-02-25T15:00:44.338 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCTDp9p5NyZJRAAVkTATRj4wmeXD1dS0Hx2nQ== 2026-02-25T15:00:44.338 INFO:teuthology.orchestra.run.trial093.stdout:[osd.10] 2026-02-25T15:00:44.338 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCUDp9pkboPDxAARS1PBlAsdqW5nnhg7EucNw== 2026-02-25T15:00:44.338 INFO:teuthology.orchestra.run.trial093.stdout:[osd.14] 2026-02-25T15:00:44.338 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCUDp9pe56zNRAA9FbMbMUOW4inYbhq8I63tA== 2026-02-25T15:00:44.338 INFO:teuthology.orchestra.run.trial093.stdout:[mgr.x] 2026-02-25T15:00:44.338 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCDDp9pXBX1MRAA8L7B4QTwmHasuEDpHX6uKw== 2026-02-25T15:00:44.338 INFO:teuthology.orchestra.run.trial093.stdout:[osd.3] 2026-02-25T15:00:44.338 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCYDp9ppJL6LxAATUMo1puxGvIvx7TFfY3dGg== 2026-02-25T15:00:44.339 INFO:teuthology.orchestra.run.trial093.stdout:[osd.7] 2026-02-25T15:00:44.339 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCZDp9pMUExGRAATGPGt5Ns8QjCij/6gh6d8g== 2026-02-25T15:00:44.339 INFO:teuthology.orchestra.run.trial093.stdout:[osd.11] 2026-02-25T15:00:44.339 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCaDp9pcDRIBBAAoa/68vOZoTPqYjAIH+Jwqg== 2026-02-25T15:00:44.339 INFO:teuthology.orchestra.run.trial093.stdout:[osd.15] 2026-02-25T15:00:44.339 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCaDp9prxeMKRAAprXwUKp/eFEq3WxZhWSO2Q== 2026-02-25T15:00:44.339 INFO:teuthology.orchestra.run.trial093.stdout:[client.0] 2026-02-25T15:00:44.339 INFO:teuthology.orchestra.run.trial093.stdout: key = AQCDDp9pbQmUNhAAGoUNoVUS8TJ7XSbHbD7w9w== 2026-02-25T15:00:44.340 INFO:teuthology.orchestra.run.trial094.stdout:[mgr.y] 2026-02-25T15:00:44.341 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCDDp9p2N0eLxAA3xJMFu/2Tp0I92UB5m6iBQ== 2026-02-25T15:00:44.341 INFO:teuthology.orchestra.run.trial094.stdout:[osd.0] 2026-02-25T15:00:44.341 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCHDp9p2qfGEhAA9xuhO9hotxkOeq2rK5aqlQ== 2026-02-25T15:00:44.341 INFO:teuthology.orchestra.run.trial094.stdout:[osd.4] 2026-02-25T15:00:44.341 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCHDp9p6StrNxAA/kgj8t5+iFcbyvx6yEZWhw== 2026-02-25T15:00:44.341 INFO:teuthology.orchestra.run.trial094.stdout:[osd.8] 2026-02-25T15:00:44.341 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCIDp9pUi/YIRAA72YP7AM2oW7n7DjnDkQCtQ== 2026-02-25T15:00:44.341 INFO:teuthology.orchestra.run.trial094.stdout:[osd.12] 2026-02-25T15:00:44.341 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCJDp9piB7DDBAABibxZ6vnVNz5HLPEH0rYzQ== 2026-02-25T15:00:44.341 INFO:teuthology.orchestra.run.trial094.stdout:[osd.1] 2026-02-25T15:00:44.341 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCNDp9p3hFKBxAA+WxSrSbsFmOoBVIWQWr4/Q== 2026-02-25T15:00:44.341 INFO:teuthology.orchestra.run.trial094.stdout:[osd.5] 2026-02-25T15:00:44.342 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCNDp9pw81gLRAAIsVcuE3wI4IFAXK2Yp42cQ== 2026-02-25T15:00:44.342 INFO:teuthology.orchestra.run.trial094.stdout:[osd.9] 2026-02-25T15:00:44.342 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCODp9pGpWNFxAAX3ilnxzAqGRV0ZSzw7mZVA== 2026-02-25T15:00:44.342 INFO:teuthology.orchestra.run.trial094.stdout:[osd.13] 2026-02-25T15:00:44.342 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCPDp9pRAoFAhAA0IkHABKTkXB9rYDDJp7CWA== 2026-02-25T15:00:44.342 INFO:teuthology.orchestra.run.trial094.stdout:[osd.2] 2026-02-25T15:00:44.342 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCTDp9p7gL0ABAALNonEfqr3o+VLJyw39SDwg== 2026-02-25T15:00:44.342 INFO:teuthology.orchestra.run.trial094.stdout:[osd.6] 2026-02-25T15:00:44.342 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCTDp9p5NyZJRAAVkTATRj4wmeXD1dS0Hx2nQ== 2026-02-25T15:00:44.342 INFO:teuthology.orchestra.run.trial094.stdout:[osd.10] 2026-02-25T15:00:44.342 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCUDp9pkboPDxAARS1PBlAsdqW5nnhg7EucNw== 2026-02-25T15:00:44.342 INFO:teuthology.orchestra.run.trial094.stdout:[osd.14] 2026-02-25T15:00:44.343 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCUDp9pe56zNRAA9FbMbMUOW4inYbhq8I63tA== 2026-02-25T15:00:44.343 INFO:teuthology.orchestra.run.trial094.stdout:[mgr.x] 2026-02-25T15:00:44.343 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCDDp9pXBX1MRAA8L7B4QTwmHasuEDpHX6uKw== 2026-02-25T15:00:44.343 INFO:teuthology.orchestra.run.trial094.stdout:[osd.3] 2026-02-25T15:00:44.343 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCYDp9ppJL6LxAATUMo1puxGvIvx7TFfY3dGg== 2026-02-25T15:00:44.343 INFO:teuthology.orchestra.run.trial094.stdout:[osd.7] 2026-02-25T15:00:44.343 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCZDp9pMUExGRAATGPGt5Ns8QjCij/6gh6d8g== 2026-02-25T15:00:44.343 INFO:teuthology.orchestra.run.trial094.stdout:[osd.11] 2026-02-25T15:00:44.343 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCaDp9pcDRIBBAAoa/68vOZoTPqYjAIH+Jwqg== 2026-02-25T15:00:44.343 INFO:teuthology.orchestra.run.trial094.stdout:[osd.15] 2026-02-25T15:00:44.343 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCaDp9prxeMKRAAprXwUKp/eFEq3WxZhWSO2Q== 2026-02-25T15:00:44.343 INFO:teuthology.orchestra.run.trial094.stdout:[client.0] 2026-02-25T15:00:44.344 INFO:teuthology.orchestra.run.trial094.stdout: key = AQCDDp9pbQmUNhAAGoUNoVUS8TJ7XSbHbD7w9w== 2026-02-25T15:00:44.344 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-25T15:00:44.379 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-25T15:00:44.382 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-25T15:00:44.466 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.508 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.510 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.549 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.588 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.590 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.629 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.666 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.669 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.708 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.746 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.749 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.789 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.825 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.830 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.868 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.902 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.910 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.949 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.980 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:44.991 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.033 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.060 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.070 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.112 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.140 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.148 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.192 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.220 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.227 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.271 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.300 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.306 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.351 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.380 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.385 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.432 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-25T15:00:45.461 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-25T15:00:45.464 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-25T15:00:45.513 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.541 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.544 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.593 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.620 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.623 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.673 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.699 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.702 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.753 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.779 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.782 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T15:00:45.832 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-25T15:00:45.859 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-25T15:00:45.862 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-25T15:00:45.912 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-25T15:00:45.913 DEBUG:teuthology.orchestra.run.trial090:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-25T15:00:45.961 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-25T15:00:46.044 DEBUG:teuthology.orchestra.run.trial090:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-25T15:00:46.109 DEBUG:teuthology.orchestra.run.trial093:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2026-02-25T15:00:46.134 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-25T15:00:46.220 DEBUG:teuthology.orchestra.run.trial093:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2026-02-25T15:00:46.245 DEBUG:teuthology.orchestra.run.trial094:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2026-02-25T15:00:46.271 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-25T15:00:46.359 DEBUG:teuthology.orchestra.run.trial094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2026-02-25T15:00:46.383 DEBUG:teuthology.orchestra.run.trial090:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:46.386 DEBUG:teuthology.orchestra.run.trial093:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:46.389 DEBUG:teuthology.orchestra.run.trial094:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-25T15:00:46.439 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-25T15:00:46.439 DEBUG:tasks.ceph_manager:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2026-02-25T15:00:46.439 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-25T15:00:46.439 DEBUG:teuthology.orchestra.run.trial090:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2026-02-25T15:00:46.442 INFO:tasks.ceph.mon.a:Started 2026-02-25T15:00:46.442 DEBUG:tasks.ceph_manager:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2026-02-25T15:00:46.442 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-25T15:00:46.443 DEBUG:teuthology.orchestra.run.trial093:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2026-02-25T15:00:46.446 INFO:tasks.ceph.mon.b:Started 2026-02-25T15:00:46.446 DEBUG:tasks.ceph_manager:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2026-02-25T15:00:46.446 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-25T15:00:46.446 DEBUG:teuthology.orchestra.run.trial094:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2026-02-25T15:00:46.481 INFO:tasks.ceph.mon.c:Started 2026-02-25T15:00:46.482 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-25T15:00:46.482 INFO:tasks.ceph.mgr.y:Restarting daemon 2026-02-25T15:00:46.482 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2026-02-25T15:00:46.484 INFO:tasks.ceph.mgr.y:Started 2026-02-25T15:00:46.484 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-25T15:00:46.484 DEBUG:teuthology.orchestra.run.trial189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2026-02-25T15:00:46.486 INFO:tasks.ceph.mgr.x:Started 2026-02-25T15:00:46.486 DEBUG:tasks.ceph:set 0 configs 2026-02-25T15:00:46.486 DEBUG:teuthology.orchestra.run.trial090:> sudo ceph --cluster ceph config dump 2026-02-25T15:00:46.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.549+0000 7f74df2fe640 10 monclient: get_monmap_and_config 2026-02-25T15:00:46.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.549+0000 7f74df2fe640 10 monclient: build_initial_monmap 2026-02-25T15:00:46.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.549+0000 7f74df2fe640 10 monclient: monmap: 2026-02-25T15:00:46.551 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:00:46.551 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:46.551 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:46.550352+0000 2026-02-25T15:00:46.551 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:46.550352+0000 2026-02-25T15:00:46.551 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:46.551 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:00:46.552 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:46.552 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:46.552 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:46.552 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:00:46.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.550+0000 7f74df2fe640 1 Processor -- start 2026-02-25T15:00:46.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.550+0000 7f74df2fe640 1 -- start start 2026-02-25T15:00:46.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.550+0000 7f74df2fe640 10 monclient: init 2026-02-25T15:00:46.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.550+0000 7f74df2fe640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:46.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.550+0000 7f74df2fe640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:00:46.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.550+0000 7f74df2fe640 10 monclient(hunting): picked mon.noname-b con 0x7f74d8058da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:46.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.550+0000 7f74df2fe640 10 monclient(hunting): picked mon.noname-c con 0x7f74d80b0410 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:46.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.550+0000 7f74df2fe640 10 monclient(hunting): picked mon.noname-a con 0x7f74d81626a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:46.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.550+0000 7f74df2fe640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d81655c0 con 0x7f74d81626a0 2026-02-25T15:00:46.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.550+0000 7f74df2fe640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d80b12b0 con 0x7f74d8058da0 2026-02-25T15:00:46.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.550+0000 7f74df2fe640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d80b1590 con 0x7f74d80b0410 2026-02-25T15:00:46.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.550+0000 7f74df2fe640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:46.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.550+0000 7f74df2fe640 10 monclient(hunting): authenticate will time out at 2936.150777s 2026-02-25T15:00:46.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.550+0000 7f74dd874640 1 -- >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74d81c6d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.90:6789/0 2026-02-25T15:00:46.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.560+0000 7f74dc872640 1 -- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74d8164eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.94:6789/0 2026-02-25T15:00:46.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.560+0000 7f74dd073640 1 -- >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74d81647a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.93:6789/0 2026-02-25T15:00:46.753 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.752+0000 7f74dd874640 1 -- >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74d81c6d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.90:6789/0 2026-02-25T15:00:46.762 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.761+0000 7f74dc872640 1 -- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74d8164eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.94:6789/0 2026-02-25T15:00:46.762 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:46.761+0000 7f74dd073640 1 -- >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74d81647a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.93:6789/0 2026-02-25T15:00:47.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:47.153+0000 7f74dd874640 1 -- >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74d81c6d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.90:6789/0 2026-02-25T15:00:47.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:47.162+0000 7f74dd073640 1 -- >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74d81647a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.93:6789/0 2026-02-25T15:00:47.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:47.163+0000 7f74dc872640 1 -- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74d8164eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.94:6789/0 2026-02-25T15:00:47.955 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:47.954+0000 7f74dd874640 1 -- >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74d81c6d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.90:6789/0 2026-02-25T15:00:47.965 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:47.964+0000 7f74dd073640 1 -- >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74d81647a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.93:6789/0 2026-02-25T15:00:47.965 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:47.964+0000 7f74dc872640 1 -- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74d8164eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.94:6789/0 2026-02-25T15:00:48.566 INFO:tasks.ceph.mon.b.trial093.stderr:--00:00:00:02.069 44974-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/90/380c19a32fd4beea3cb0388d22417b25838f9e.debug 2026-02-25T15:00:48.568 INFO:tasks.ceph.mon.a.trial090.stderr:--00:00:00:02.075 45499-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/90/380c19a32fd4beea3cb0388d22417b25838f9e.debug 2026-02-25T15:00:48.590 INFO:tasks.ceph.mon.c.trial094.stderr:--00:00:00:02.060 44834-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/90/380c19a32fd4beea3cb0388d22417b25838f9e.debug 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74c6ffd640 10 monclient(hunting): tick 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74c6ffd640 10 monclient(hunting): _check_auth_tickets 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74c6ffd640 1 monclient(hunting): continuing hunt 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74c6ffd640 10 monclient(hunting): _reopen_session rank -1 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74c6ffd640 1 -- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74d8164eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74c6ffd640 1 -- >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74d81647a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74c6ffd640 1 -- >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74d81c6d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74c6ffd640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74c6ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f74d80b0410 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74c6ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f74d81626a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74c6ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f74d8058da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74c6ffd640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d80b1870 con 0x7f74d81626a0 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74dc872640 1 -- >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74bc044fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.90:6789/0 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74c6ffd640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d81655c0 con 0x7f74d8058da0 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74c6ffd640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d80b12b0 con 0x7f74d80b0410 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74c6ffd640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74dd874640 1 -- >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74bc066880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.93:6789/0 2026-02-25T15:00:49.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.550+0000 7f74dd073640 1 -- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74bc0237c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.94:6789/0 2026-02-25T15:00:49.752 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.751+0000 7f74dc872640 1 -- >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74bc044fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.90:6789/0 2026-02-25T15:00:49.752 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.751+0000 7f74dd874640 1 -- >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74bc066880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.93:6789/0 2026-02-25T15:00:49.752 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:49.751+0000 7f74dd073640 1 -- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74bc0237c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.94:6789/0 2026-02-25T15:00:50.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:50.152+0000 7f74dd874640 1 -- >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74bc066880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.93:6789/0 2026-02-25T15:00:50.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:50.152+0000 7f74dc872640 1 -- >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74bc044fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.90:6789/0 2026-02-25T15:00:50.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:50.152+0000 7f74dd073640 1 -- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74bc0237c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.94:6789/0 2026-02-25T15:00:50.954 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:50.953+0000 7f74dd874640 1 -- >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74bc066880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.93:6789/0 2026-02-25T15:00:50.954 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:50.953+0000 7f74dc872640 1 -- >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74bc044fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.90:6789/0 2026-02-25T15:00:50.954 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:50.953+0000 7f74dd073640 1 -- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74bc0237c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.94:6789/0 2026-02-25T15:00:52.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.550+0000 7f74c6ffd640 10 monclient(hunting): tick 2026-02-25T15:00:52.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.550+0000 7f74c6ffd640 10 monclient(hunting): _check_auth_tickets 2026-02-25T15:00:52.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.550+0000 7f74c6ffd640 1 monclient(hunting): continuing hunt 2026-02-25T15:00:52.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.550+0000 7f74c6ffd640 10 monclient(hunting): _reopen_session rank -1 2026-02-25T15:00:52.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.550+0000 7f74c6ffd640 1 -- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74bc0237c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-25T15:00:52.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.550+0000 7f74c6ffd640 1 -- >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74bc066880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-25T15:00:52.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.550+0000 7f74c6ffd640 1 -- >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74bc044fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-25T15:00:52.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.550+0000 7f74c6ffd640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:00:52.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.550+0000 7f74c6ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f74d80b0410 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:52.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.550+0000 7f74c6ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f74d81626a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:52.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.550+0000 7f74c6ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f74d8058da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:52.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.550+0000 7f74c6ffd640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d80b1590 con 0x7f74d81626a0 2026-02-25T15:00:52.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.550+0000 7f74c6ffd640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d80b1870 con 0x7f74d8058da0 2026-02-25T15:00:52.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.551+0000 7f74c6ffd640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d81655c0 con 0x7f74d80b0410 2026-02-25T15:00:52.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.551+0000 7f74c6ffd640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:52.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.551+0000 7f74dc872640 1 -- >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74bc044f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.90:6789/0 2026-02-25T15:00:52.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.551+0000 7f74dd874640 1 -- >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74bc066880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.93:6789/0 2026-02-25T15:00:52.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.551+0000 7f74dd073640 1 -- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74bc0237c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.94:6789/0 2026-02-25T15:00:52.753 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.752+0000 7f74dd874640 1 -- >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74bc066880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.93:6789/0 2026-02-25T15:00:52.754 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.752+0000 7f74dc872640 1 -- >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74bc044f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.90:6789/0 2026-02-25T15:00:52.754 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:52.752+0000 7f74dd073640 1 -- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74bc0237c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.94:6789/0 2026-02-25T15:00:52.822 INFO:tasks.ceph.mon.c.trial094.stderr:--00:00:00:06.291 44834-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/74/cc669080706d0b703f0059c774676798cfdb1f.debug 2026-02-25T15:00:52.829 INFO:tasks.ceph.mon.b.trial093.stderr:--00:00:00:06.332 44974-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/74/cc669080706d0b703f0059c774676798cfdb1f.debug 2026-02-25T15:00:52.851 INFO:tasks.ceph.mon.a.trial090.stderr:--00:00:00:06.358 45499-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/74/cc669080706d0b703f0059c774676798cfdb1f.debug 2026-02-25T15:00:53.155 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:53.153+0000 7f74dd874640 1 -- >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74bc066880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.93:6789/0 2026-02-25T15:00:53.155 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:53.153+0000 7f74dc872640 1 -- >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74bc044f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.90:6789/0 2026-02-25T15:00:53.155 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:53.153+0000 7f74dd073640 1 -- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74bc0237c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.94:6789/0 2026-02-25T15:00:53.955 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:53.954+0000 7f74dd874640 1 -- >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74bc066880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.93:6789/0 2026-02-25T15:00:53.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:53.954+0000 7f74dc872640 1 -- >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74bc044f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.90:6789/0 2026-02-25T15:00:53.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:53.954+0000 7f74dd073640 1 -- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74bc0237c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.94:6789/0 2026-02-25T15:00:54.331 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:00:07.800 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:00:54.337 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:00:07.839 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:00:54.362 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:00:07.869 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:00:54.481 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:00:07.950 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:00:54.486 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:00:07.988 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:00:54.511 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:00:08.018 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:00:55.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.551+0000 7f74c6ffd640 10 monclient(hunting): tick 2026-02-25T15:00:55.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.551+0000 7f74c6ffd640 10 monclient(hunting): _check_auth_tickets 2026-02-25T15:00:55.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.551+0000 7f74c6ffd640 1 monclient(hunting): continuing hunt 2026-02-25T15:00:55.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.551+0000 7f74c6ffd640 10 monclient(hunting): _reopen_session rank -1 2026-02-25T15:00:55.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.551+0000 7f74c6ffd640 1 -- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74bc0237c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-25T15:00:55.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.551+0000 7f74c6ffd640 1 -- >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74bc066880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-25T15:00:55.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.551+0000 7f74c6ffd640 1 -- >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74bc044f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-25T15:00:55.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.551+0000 7f74c6ffd640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:00:55.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.551+0000 7f74c6ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f74d8058da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:55.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.551+0000 7f74c6ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f74d80b0410 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:55.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.551+0000 7f74c6ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f74d81626a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:55.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.551+0000 7f74c6ffd640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d80b12b0 con 0x7f74d81626a0 2026-02-25T15:00:55.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.551+0000 7f74c6ffd640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d80b1590 con 0x7f74d8058da0 2026-02-25T15:00:55.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.551+0000 7f74c6ffd640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d80b1870 con 0x7f74d80b0410 2026-02-25T15:00:55.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.551+0000 7f74c6ffd640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:55.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.563+0000 7f74dc872640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 0x7f74bc0237c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:52516/0 (socket says 10.20.193.90:52516) 2026-02-25T15:00:55.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:55.563+0000 7f74dc872640 1 -- 10.20.193.90:0/3022482678 learned_addr learned my addr 10.20.193.90:0/3022482678 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:00:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.678+0000 7f74c7fff640 1 -- 10.20.193.90:0/3022482678 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1566710153 0 0) 0x7f74d80b12b0 con 0x7f74d81626a0 2026-02-25T15:00:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.678+0000 7f74c7fff640 10 monclient(hunting): _negotiate 2026-02-25T15:00:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.678+0000 7f74c7fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.678+0000 7f74c7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.678+0000 7f74c7fff640 10 monclient(hunting): my global_id is 4101 2026-02-25T15:00:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.678+0000 7f74c7fff640 1 -- 10.20.193.90:0/3022482678 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74c0003610 con 0x7f74d81626a0 2026-02-25T15:00:56.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.685+0000 7f74c7fff640 1 -- 10.20.193.90:0/3022482678 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2116610017 0 0) 0x7f74d80b1590 con 0x7f74d8058da0 2026-02-25T15:00:56.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.685+0000 7f74c7fff640 10 monclient(hunting): _negotiate 2026-02-25T15:00:56.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.685+0000 7f74c7fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:56.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.685+0000 7f74c7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:56.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.685+0000 7f74c7fff640 10 monclient(hunting): my global_id is 4105 2026-02-25T15:00:56.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.685+0000 7f74c7fff640 1 -- 10.20.193.90:0/3022482678 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74d80b12b0 con 0x7f74d8058da0 2026-02-25T15:00:56.700 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.699+0000 7f74c7fff640 1 -- 10.20.193.90:0/3022482678 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2502548077 0 0) 0x7f74d80b1870 con 0x7f74d80b0410 2026-02-25T15:00:56.700 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.699+0000 7f74c7fff640 10 monclient(hunting): _negotiate 2026-02-25T15:00:56.700 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.699+0000 7f74c7fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:56.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.699+0000 7f74c7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:56.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.699+0000 7f74c7fff640 10 monclient(hunting): my global_id is 4106 2026-02-25T15:00:56.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.699+0000 7f74c7fff640 1 -- 10.20.193.90:0/3022482678 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74d80b1590 con 0x7f74d80b0410 2026-02-25T15:00:56.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74c7fff640 1 -- 10.20.193.90:0/3022482678 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 917348397 0 0) 0x7f74c0003610 con 0x7f74d81626a0 2026-02-25T15:00:56.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74c7fff640 1 -- 10.20.193.90:0/3022482678 >> v1:10.20.193.94:6789/0 conn(0x7f74d80b0410 legacy=0x7f74bc0237c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:56.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74c7fff640 1 -- 10.20.193.90:0/3022482678 >> v1:10.20.193.93:6789/0 conn(0x7f74d8058da0 legacy=0x7f74bc066800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:56.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74c7fff640 10 monclient: _finish_hunting 0 2026-02-25T15:00:56.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74c7fff640 1 monclient: found mon.noname-a 2026-02-25T15:00:56.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74c7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:56.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74c7fff640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:00:56.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74c7fff640 1 -- 10.20.193.90:0/3022482678 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d81655c0 con 0x7f74d81626a0 2026-02-25T15:00:56.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74c7fff640 10 monclient: _finish_auth 0 2026-02-25T15:00:56.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74c7fff640 10 monclient: _check_auth_tickets 2026-02-25T15:00:56.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:56.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74c7fff640 1 -- 10.20.193.90:0/3022482678 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f74d40044f0 con 0x7f74d81626a0 2026-02-25T15:00:56.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74c7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:56.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74c7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:00:56.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74c7fff640 10 monclient: dump: 2026-02-25T15:00:56.711 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:00:56.712 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:56.712 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:56.712 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:56.712 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:56.712 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:00:56.712 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:56.712 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:56.712 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:56.712 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:00:56.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74df2fe640 5 monclient: authenticate success, global_id 4101 2026-02-25T15:00:56.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.708+0000 7f74df2fe640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:00:56.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.738+0000 7f74c7fff640 1 -- 10.20.193.90:0/3022482678 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f74d4004700 con 0x7f74d81626a0 2026-02-25T15:00:56.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.738+0000 7f74c7fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:56.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.738+0000 7f74c7fff640 1 -- 10.20.193.90:0/3022482678 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f74d4004b40 con 0x7f74d81626a0 2026-02-25T15:00:56.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.738+0000 7f74c7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:56.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.738+0000 7f74c7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:56.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.738+0000 7f74c7fff640 10 monclient: dump: 2026-02-25T15:00:56.740 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:00:56.740 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:56.741 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:56.741 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:56.741 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:56.741 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:00:56.741 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:56.741 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:56.741 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:56.741 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:00:56.741 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.738+0000 7f74df2fe640 10 monclient: get_monmap_and_config success 2026-02-25T15:00:56.741 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.738+0000 7f74df2fe640 4 set_mon_vals no callback set 2026-02-25T15:00:56.741 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.738+0000 7f74df2fe640 10 monclient: shutdown 2026-02-25T15:00:56.741 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.738+0000 7f74df2fe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:56.742 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.738+0000 7f74df2fe640 1 -- 10.20.193.90:0/3022482678 >> v1:10.20.193.90:6789/0 conn(0x7f74d81626a0 legacy=0x7f74bc044f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:56.742 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.738+0000 7f74df2fe640 1 -- 10.20.193.90:0/3022482678 shutdown_connections 2026-02-25T15:00:56.742 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.738+0000 7f74df2fe640 1 -- 10.20.193.90:0/3022482678 >> 10.20.193.90:0/3022482678 conn(0x7f74d80b65f0 msgr2=0x7f74d80b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:56.742 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.738+0000 7f74df2fe640 1 -- 10.20.193.90:0/3022482678 shutdown_connections 2026-02-25T15:00:56.742 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.738+0000 7f74df2fe640 1 -- 10.20.193.90:0/3022482678 wait complete. 2026-02-25T15:00:56.742 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 10 monclient: build_initial_monmap 2026-02-25T15:00:56.742 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 10 monclient: monmap: 2026-02-25T15:00:56.742 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:00:56.742 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:00:56.742 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:00:56.742 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:00:56.742 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:56.743 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:00:56.743 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:56.743 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:56.743 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:56.743 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:00:56.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 1 librados: starting msgr at 2026-02-25T15:00:56.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 1 librados: starting objecter 2026-02-25T15:00:56.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:00:56.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 1 Processor -- start 2026-02-25T15:00:56.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 1 -- start start 2026-02-25T15:00:56.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 1 librados: setting wanted keys 2026-02-25T15:00:56.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 1 librados: calling monclient init 2026-02-25T15:00:56.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 10 monclient: init 2026-02-25T15:00:56.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:56.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:00:56.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 10 monclient(hunting): picked mon.noname-a con 0x7f74d8058da0 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:56.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 10 monclient(hunting): picked mon.noname-b con 0x7f74d80b0410 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:56.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 10 monclient(hunting): picked mon.noname-c con 0x7f74d81626a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:56.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d8057410 con 0x7f74d8058da0 2026-02-25T15:00:56.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d805b890 con 0x7f74d80b0410 2026-02-25T15:00:56.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d805bb70 con 0x7f74d81626a0 2026-02-25T15:00:56.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:56.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.739+0000 7f74df2fe640 10 monclient(hunting): authenticate will time out at 2946.339988s 2026-02-25T15:00:56.747 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.744+0000 7f74dd874640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f74d81626a0 0x7f74d805cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:52538/0 (socket says 10.20.193.90:52538) 2026-02-25T15:00:56.747 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.744+0000 7f74dd874640 1 -- 10.20.193.90:0/3638383390 learned_addr learned my addr 10.20.193.90:0/3638383390 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:00:56.747 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.744+0000 7f74c5ffb640 10 client.?.objecter ms_handle_connect 0x7f74d81626a0 2026-02-25T15:00:56.747 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.744+0000 7f74c5ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:56.747 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.744+0000 7f74c5ffb640 10 client.?.objecter ms_handle_connect 0x7f74d80b0410 2026-02-25T15:00:56.747 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.744+0000 7f74c5ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:56.749 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.748+0000 7f74c5ffb640 10 client.?.objecter ms_handle_connect 0x7f74d8058da0 2026-02-25T15:00:56.750 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.748+0000 7f74c5ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:56.782 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.781+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 335125947 0 0) 0x7f74d8057410 con 0x7f74d8058da0 2026-02-25T15:00:56.783 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.781+0000 7f74c5ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:00:56.783 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.781+0000 7f74c5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:56.783 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.781+0000 7f74c5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:56.783 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.781+0000 7f74c5ffb640 10 monclient(hunting): my global_id is 4113 2026-02-25T15:00:56.783 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.781+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74a8003610 con 0x7f74d8058da0 2026-02-25T15:00:56.787 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.786+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2703570372 0 0) 0x7f74d805bb70 con 0x7f74d81626a0 2026-02-25T15:00:56.787 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.786+0000 7f74c5ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:00:56.787 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.786+0000 7f74c5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:56.788 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.786+0000 7f74c5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:56.788 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.786+0000 7f74c5ffb640 10 monclient(hunting): my global_id is 4112 2026-02-25T15:00:56.788 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.786+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74d8057410 con 0x7f74d81626a0 2026-02-25T15:00:56.788 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.787+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 390495473 0 0) 0x7f74d805b890 con 0x7f74d80b0410 2026-02-25T15:00:56.788 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.787+0000 7f74c5ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:00:56.789 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.787+0000 7f74c5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:56.789 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.787+0000 7f74c5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:56.789 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.787+0000 7f74c5ffb640 10 monclient(hunting): my global_id is 4111 2026-02-25T15:00:56.789 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.787+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74d805bb70 con 0x7f74d80b0410 2026-02-25T15:00:56.794 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.792+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1978644505 0 0) 0x7f74a8003610 con 0x7f74d8058da0 2026-02-25T15:00:56.794 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.792+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f74d805b890 con 0x7f74d8058da0 2026-02-25T15:00:56.794 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.792+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f74c8004280 con 0x7f74d8058da0 2026-02-25T15:00:56.794 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.792+0000 7f74c5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:56.794 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.792+0000 7f74c5ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:00:56.794 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.792+0000 7f74c5ffb640 10 monclient(hunting): dump: 2026-02-25T15:00:56.794 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:00:56.794 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:56.794 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:56.795 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:56.795 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:56.795 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:00:56.795 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:56.795 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:56.795 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:56.795 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:00:56.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.792+0000 7f74c5ffb640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:00:56.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1572641098 0 0) 0x7f74d805b890 con 0x7f74d8058da0 2026-02-25T15:00:56.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 >> v1:10.20.193.94:6789/0 conn(0x7f74d81626a0 legacy=0x7f74d805cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:56.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 >> v1:10.20.193.93:6789/0 conn(0x7f74d80b0410 legacy=0x7f74d8056d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:56.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74c5ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:00:56.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74c5ffb640 1 monclient: found mon.a 2026-02-25T15:00:56.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74c5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:56.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74c5ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:56.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d805be50 con 0x7f74d8058da0 2026-02-25T15:00:56.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74c5ffb640 10 monclient: _finish_auth 0 2026-02-25T15:00:56.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74c5ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:00:56.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:56.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74df2fe640 5 monclient: authenticate success, global_id 4113 2026-02-25T15:00:56.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74df2fe640 10 monclient: _renew_subs 2026-02-25T15:00:56.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74df2fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:56.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74df2fe640 1 -- 10.20.193.90:0/3638383390 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f74d805c130 con 0x7f74d8058da0 2026-02-25T15:00:56.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74df2fe640 10 client.4113.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:00:56.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74df2fe640 10 monclient: _renew_subs 2026-02-25T15:00:56.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74df2fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:56.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74df2fe640 1 -- 10.20.193.90:0/3638383390 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f74d805d660 con 0x7f74d8058da0 2026-02-25T15:00:56.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74df2fe640 1 librados: init done 2026-02-25T15:00:56.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74df2fe640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:56.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74df2fe640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:56.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74df2fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:56.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.824+0000 7f74df2fe640 1 -- 10.20.193.90:0/3638383390 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d813f360 con 0x7f74d8058da0 2026-02-25T15:00:56.922 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.920+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f74c8004490 con 0x7f74d8058da0 2026-02-25T15:00:56.922 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.920+0000 7f74c5ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:56.923 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.920+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f74c8004b40 con 0x7f74d8058da0 2026-02-25T15:00:56.923 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.920+0000 7f74c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:56.923 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.920+0000 7f74c5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:56.923 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.920+0000 7f74c5ffb640 10 monclient: dump: 2026-02-25T15:00:56.923 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:00:56.923 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:56.923 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:56.923 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:56.923 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:56.923 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:00:56.923 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:56.923 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:56.924 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:56.924 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:00:56.924 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.920+0000 7f74c77fe640 4 set_mon_vals no callback set 2026-02-25T15:00:56.924 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.920+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 1) ==== 939+0+0 (unknown 246333160 0 0) 0x7f74c8005130 con 0x7f74d8058da0 2026-02-25T15:00:56.924 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.920+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 720+0+0 (unknown 1665658004 0 0) 0x7f74c8006bb0 con 0x7f74d8058da0 2026-02-25T15:00:56.924 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.920+0000 7f74c5ffb640 10 client.4113.objecter ms_dispatch2 0x7f74d8000b90 osd_map(1..1 src has 1..1) 2026-02-25T15:00:56.924 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.920+0000 7f74c5ffb640 3 client.4113.objecter handle_osd_map got epochs [1,1] > 0 2026-02-25T15:00:56.924 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.920+0000 7f74c5ffb640 3 client.4113.objecter handle_osd_map decoding full epoch 1 2026-02-25T15:00:56.924 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.920+0000 7f74c5ffb640 20 client.4113.objecter dump_active .. 0 homeless 2026-02-25T15:00:56.924 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.921+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77519 (unknown 1092875540 0 440498749) 0x7f74c8019f20 con 0x7f74d8058da0 2026-02-25T15:00:56.924 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.921+0000 7f74c5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:56.924 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.921+0000 7f74c5ffb640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:00:56.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.950+0000 7f74df2fe640 10 start_mon_command cmd=[{"prefix": "config dump"}] 2026-02-25T15:00:56.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.950+0000 7f74df2fe640 10 monclient: _send_command 2 [{"prefix": "config dump"}] 2026-02-25T15:00:56.952 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.950+0000 7f74df2fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:56.952 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.950+0000 7f74df2fe640 1 -- 10.20.193.90:0/3638383390 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "config dump"} v 0) -- 0x7f74d80b29e0 con 0x7f74d8058da0 2026-02-25T15:00:56.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.993+0000 7f74c5ffb640 1 -- 10.20.193.90:0/3638383390 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "config dump"}]=0 v1) ==== 59+0+36 (unknown 4069917974 0 2919340917) 0x7f74c8005630 con 0x7f74d8058da0 2026-02-25T15:00:56.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.993+0000 7f74c5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config dump"}] 2026-02-25T15:00:56.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.993+0000 7f74c5ffb640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:00:56.995 INFO:teuthology.orchestra.run.trial090.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-25T15:00:56.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.994+0000 7f74df2fe640 10 librados: watch_flush enter 2026-02-25T15:00:56.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.994+0000 7f74df2fe640 10 librados: watch_flush exit 2026-02-25T15:00:56.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.994+0000 7f74df2fe640 20 client.4113.objecter shutdown clearing up homeless session... 2026-02-25T15:00:56.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.994+0000 7f74df2fe640 10 client.4113.objecter successfully canceled tick 2026-02-25T15:00:56.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.994+0000 7f74df2fe640 10 monclient: shutdown 2026-02-25T15:00:56.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.994+0000 7f74df2fe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:56.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.994+0000 7f74df2fe640 1 -- 10.20.193.90:0/3638383390 >> v1:10.20.193.90:6789/0 conn(0x7f74d8058da0 legacy=0x7f74d8141de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:56.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.994+0000 7f74df2fe640 1 -- 10.20.193.90:0/3638383390 shutdown_connections 2026-02-25T15:00:56.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.994+0000 7f74df2fe640 1 -- 10.20.193.90:0/3638383390 >> 10.20.193.90:0/3638383390 conn(0x7f74d80b65f0 msgr2=0x7f74d80a7840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:56.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.994+0000 7f74df2fe640 1 -- 10.20.193.90:0/3638383390 shutdown_connections 2026-02-25T15:00:56.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.994+0000 7f74df2fe640 1 -- 10.20.193.90:0/3638383390 wait complete. 2026-02-25T15:00:56.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:56.994+0000 7f74df2fe640 1 librados: shutdown 2026-02-25T15:00:57.004 INFO:tasks.ceph:Setting crush tunables to default 2026-02-25T15:00:57.004 DEBUG:teuthology.orchestra.run.trial090:> sudo ceph --cluster ceph osd crush tunables default 2026-02-25T15:00:57.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.068+0000 7fd9abb8a640 10 monclient: get_monmap_and_config 2026-02-25T15:00:57.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.068+0000 7fd9abb8a640 10 monclient: build_initial_monmap 2026-02-25T15:00:57.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.068+0000 7fd9abb8a640 10 monclient: monmap: 2026-02-25T15:00:57.070 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:00:57.070 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:57.070 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:57.069365+0000 2026-02-25T15:00:57.070 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:57.069365+0000 2026-02-25T15:00:57.070 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:57.071 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:00:57.071 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:57.071 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:57.071 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:57.071 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:00:57.073 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.069+0000 7fd9abb8a640 1 Processor -- start 2026-02-25T15:00:57.073 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.069+0000 7fd9abb8a640 1 -- start start 2026-02-25T15:00:57.073 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.069+0000 7fd9abb8a640 10 monclient: init 2026-02-25T15:00:57.073 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.069+0000 7fd9abb8a640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:57.073 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.069+0000 7fd9abb8a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:00:57.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.069+0000 7fd9abb8a640 10 monclient(hunting): picked mon.noname-c con 0x7fd9a415afd0 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:57.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.069+0000 7fd9abb8a640 10 monclient(hunting): picked mon.noname-b con 0x7fd9a415b3a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:57.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.069+0000 7fd9abb8a640 10 monclient(hunting): picked mon.noname-a con 0x7fd9a4163890 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:57.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.069+0000 7fd9abb8a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a4162380 con 0x7fd9a4163890 2026-02-25T15:00:57.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.069+0000 7fd9abb8a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a40b1c30 con 0x7fd9a415b3a0 2026-02-25T15:00:57.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.069+0000 7fd9abb8a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a40b1f10 con 0x7fd9a415afd0 2026-02-25T15:00:57.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.069+0000 7fd9abb8a640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:57.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.069+0000 7fd9abb8a640 10 monclient(hunting): authenticate will time out at 2946.669792s 2026-02-25T15:00:57.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.070+0000 7fd9aa100640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fd9a4163890 0x7fd9a4164480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:48614/0 (socket says 10.20.193.90:48614) 2026-02-25T15:00:57.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.070+0000 7fd9aa100640 1 -- 10.20.193.90:0/1817740015 learned_addr learned my addr 10.20.193.90:0/1817740015 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:00:57.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.071+0000 7fd9a88fd640 1 -- 10.20.193.90:0/1817740015 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3678099319 0 0) 0x7fd9a4162380 con 0x7fd9a4163890 2026-02-25T15:00:57.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.071+0000 7fd9a88fd640 10 monclient(hunting): _negotiate 2026-02-25T15:00:57.075 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.071+0000 7fd9a88fd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:57.075 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.071+0000 7fd9a88fd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:57.075 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.071+0000 7fd9a88fd640 10 monclient(hunting): my global_id is 4116 2026-02-25T15:00:57.075 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.071+0000 7fd9a88fd640 1 -- 10.20.193.90:0/1817740015 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd98c003610 con 0x7fd9a4163890 2026-02-25T15:00:57.077 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9a88fd640 1 -- 10.20.193.90:0/1817740015 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2792723056 0 0) 0x7fd98c003610 con 0x7fd9a4163890 2026-02-25T15:00:57.077 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9a88fd640 1 -- 10.20.193.90:0/1817740015 >> v1:10.20.193.94:6789/0 conn(0x7fd9a415afd0 legacy=0x7fd9a4059be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:00:57.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9a88fd640 1 -- 10.20.193.90:0/1817740015 >> v1:10.20.193.93:6789/0 conn(0x7fd9a415b3a0 legacy=0x7fd9a405d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:00:57.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9a88fd640 10 monclient: _finish_hunting 0 2026-02-25T15:00:57.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9a88fd640 1 monclient: found mon.noname-a 2026-02-25T15:00:57.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9a88fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:57.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9a88fd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:00:57.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9a88fd640 1 -- 10.20.193.90:0/1817740015 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a40b21f0 con 0x7fd9a4163890 2026-02-25T15:00:57.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9a88fd640 10 monclient: _finish_auth 0 2026-02-25T15:00:57.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9a88fd640 10 monclient: _check_auth_tickets 2026-02-25T15:00:57.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9a88fd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:57.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9a88fd640 1 -- 10.20.193.90:0/1817740015 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd9a0003170 con 0x7fd9a4163890 2026-02-25T15:00:57.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9a88fd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:57.079 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9a88fd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:00:57.079 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9a88fd640 10 monclient: dump: 2026-02-25T15:00:57.079 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:00:57.079 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:57.079 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:57.079 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:57.079 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:57.079 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:00:57.079 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:57.079 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:57.079 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:57.079 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:00:57.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9abb8a640 5 monclient: authenticate success, global_id 4116 2026-02-25T15:00:57.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.073+0000 7fd9abb8a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:00:57.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.075+0000 7fd9a88fd640 1 -- 10.20.193.90:0/1817740015 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd9a0003380 con 0x7fd9a4163890 2026-02-25T15:00:57.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.075+0000 7fd9a88fd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:57.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.075+0000 7fd9a88fd640 1 -- 10.20.193.90:0/1817740015 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd9a00037c0 con 0x7fd9a4163890 2026-02-25T15:00:57.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.075+0000 7fd9a88fd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:57.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.075+0000 7fd9a88fd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:57.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.075+0000 7fd9a88fd640 10 monclient: dump: 2026-02-25T15:00:57.080 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:00:57.080 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:57.080 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:57.080 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:57.081 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:57.081 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:00:57.081 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:57.081 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:57.081 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:57.081 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:00:57.081 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.075+0000 7fd9abb8a640 10 monclient: get_monmap_and_config success 2026-02-25T15:00:57.081 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.075+0000 7fd9abb8a640 4 set_mon_vals no callback set 2026-02-25T15:00:57.081 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.075+0000 7fd9abb8a640 10 monclient: shutdown 2026-02-25T15:00:57.081 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.075+0000 7fd9abb8a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:57.081 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.075+0000 7fd9abb8a640 1 -- 10.20.193.90:0/1817740015 >> v1:10.20.193.90:6789/0 conn(0x7fd9a4163890 legacy=0x7fd9a4164480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:57.081 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.076+0000 7fd9abb8a640 1 -- 10.20.193.90:0/1817740015 shutdown_connections 2026-02-25T15:00:57.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.076+0000 7fd9abb8a640 1 -- 10.20.193.90:0/1817740015 >> 10.20.193.90:0/1817740015 conn(0x7fd9a40b2f10 msgr2=0x7fd9a40b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:57.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.076+0000 7fd9abb8a640 1 -- 10.20.193.90:0/1817740015 shutdown_connections 2026-02-25T15:00:57.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.076+0000 7fd9abb8a640 1 -- 10.20.193.90:0/1817740015 wait complete. 2026-02-25T15:00:57.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.076+0000 7fd9abb8a640 10 monclient: build_initial_monmap 2026-02-25T15:00:57.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.076+0000 7fd9abb8a640 10 monclient: monmap: 2026-02-25T15:00:57.082 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:00:57.082 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:00:57.082 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:00:57.082 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:00:57.082 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:57.082 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:00:57.082 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:57.082 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:57.083 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:57.083 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:00:57.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 1 librados: starting msgr at 2026-02-25T15:00:57.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 1 librados: starting objecter 2026-02-25T15:00:57.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:00:57.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 1 Processor -- start 2026-02-25T15:00:57.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 1 -- start start 2026-02-25T15:00:57.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 1 librados: setting wanted keys 2026-02-25T15:00:57.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 1 librados: calling monclient init 2026-02-25T15:00:57.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 10 monclient: init 2026-02-25T15:00:57.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:57.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:00:57.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 10 monclient(hunting): picked mon.noname-b con 0x7fd9a415afd0 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:57.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 10 monclient(hunting): picked mon.noname-c con 0x7fd9a415b3a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:57.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 10 monclient(hunting): picked mon.noname-a con 0x7fd9a4163890 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:57.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a411c460 con 0x7fd9a4163890 2026-02-25T15:00:57.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a411c740 con 0x7fd9a415afd0 2026-02-25T15:00:57.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a411ca20 con 0x7fd9a415b3a0 2026-02-25T15:00:57.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:57.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.077+0000 7fd9abb8a640 10 monclient(hunting): authenticate will time out at 2946.677791s 2026-02-25T15:00:57.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.078+0000 7fd9a90fe640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fd9a415b3a0 0x7fd9a411bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:52588/0 (socket says 10.20.193.90:52588) 2026-02-25T15:00:57.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.078+0000 7fd9a90fe640 1 -- 10.20.193.90:0/1993942047 learned_addr learned my addr 10.20.193.90:0/1993942047 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:00:57.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.078+0000 7fd9927fc640 10 client.?.objecter ms_handle_connect 0x7fd9a415b3a0 2026-02-25T15:00:57.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.078+0000 7fd9927fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:57.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.078+0000 7fd9927fc640 10 client.?.objecter ms_handle_connect 0x7fd9a415afd0 2026-02-25T15:00:57.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.078+0000 7fd9927fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:57.087 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.078+0000 7fd9927fc640 10 client.?.objecter ms_handle_connect 0x7fd9a4163890 2026-02-25T15:00:57.087 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.078+0000 7fd9927fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:57.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.092+0000 7fd9927fc640 1 -- 10.20.193.90:0/1993942047 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2286299895 0 0) 0x7fd9a411c460 con 0x7fd9a4163890 2026-02-25T15:00:57.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.092+0000 7fd9927fc640 10 monclient(hunting): _negotiate 2026-02-25T15:00:57.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.092+0000 7fd9927fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:57.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.092+0000 7fd9927fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:57.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.092+0000 7fd9927fc640 10 monclient(hunting): my global_id is 4119 2026-02-25T15:00:57.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.092+0000 7fd9927fc640 1 -- 10.20.193.90:0/1993942047 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd974003610 con 0x7fd9a4163890 2026-02-25T15:00:57.097 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.095+0000 7fd9927fc640 1 -- 10.20.193.90:0/1993942047 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 531181158 0 0) 0x7fd974003610 con 0x7fd9a4163890 2026-02-25T15:00:57.097 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.095+0000 7fd9927fc640 1 -- 10.20.193.90:0/1993942047 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd9a411c460 con 0x7fd9a4163890 2026-02-25T15:00:57.097 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.095+0000 7fd9927fc640 1 -- 10.20.193.90:0/1993942047 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd9a00036d0 con 0x7fd9a4163890 2026-02-25T15:00:57.097 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.095+0000 7fd9927fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:57.097 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.095+0000 7fd9927fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:00:57.097 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.095+0000 7fd9927fc640 10 monclient(hunting): dump: 2026-02-25T15:00:57.097 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:00:57.097 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:57.097 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:57.097 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:57.098 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:57.098 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:00:57.098 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:57.098 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:57.098 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:57.098 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:00:57.098 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.095+0000 7fd9927fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:00:57.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9927fc640 1 -- 10.20.193.90:0/1993942047 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4190887698 0 0) 0x7fd9a411c460 con 0x7fd9a4163890 2026-02-25T15:00:57.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9927fc640 1 -- 10.20.193.90:0/1993942047 >> v1:10.20.193.94:6789/0 conn(0x7fd9a415b3a0 legacy=0x7fd9a411bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:57.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9927fc640 1 -- 10.20.193.90:0/1993942047 >> v1:10.20.193.93:6789/0 conn(0x7fd9a415afd0 legacy=0x7fd9a405d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:57.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9927fc640 10 monclient: _finish_hunting 0 2026-02-25T15:00:57.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9927fc640 1 monclient: found mon.a 2026-02-25T15:00:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9927fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9927fc640 1 -- 10.20.193.90:0/1993942047 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a413f620 con 0x7fd9a4163890 2026-02-25T15:00:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9927fc640 10 monclient: _finish_auth 0 2026-02-25T15:00:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9927fc640 10 monclient: _check_auth_tickets 2026-02-25T15:00:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9abb8a640 5 monclient: authenticate success, global_id 4119 2026-02-25T15:00:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9abb8a640 10 monclient: _renew_subs 2026-02-25T15:00:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9abb8a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9abb8a640 1 -- 10.20.193.90:0/1993942047 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a413f900 con 0x7fd9a4163890 2026-02-25T15:00:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9abb8a640 10 client.4119.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:00:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9abb8a640 10 monclient: _renew_subs 2026-02-25T15:00:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9abb8a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9abb8a640 1 -- 10.20.193.90:0/1993942047 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd9a4114290 con 0x7fd9a4163890 2026-02-25T15:00:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.097+0000 7fd9abb8a640 1 librados: init done 2026-02-25T15:00:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.098+0000 7fd9abb8a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.098+0000 7fd9abb8a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.098+0000 7fd9abb8a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.098+0000 7fd9abb8a640 1 -- 10.20.193.90:0/1993942047 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a40b3e80 con 0x7fd9a4163890 2026-02-25T15:00:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.100+0000 7fd9927fc640 1 -- 10.20.193.90:0/1993942047 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd9a0003380 con 0x7fd9a4163890 2026-02-25T15:00:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.100+0000 7fd9927fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.100+0000 7fd9927fc640 1 -- 10.20.193.90:0/1993942047 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd9a00050a0 con 0x7fd9a4163890 2026-02-25T15:00:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.100+0000 7fd9927fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.100+0000 7fd9927fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.100+0000 7fd9927fc640 10 monclient: dump: 2026-02-25T15:00:57.104 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:00:57.104 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:57.104 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:57.104 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:57.104 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:57.104 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:00:57.104 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:57.104 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:57.104 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:57.104 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:00:57.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.100+0000 7fd993fff640 4 set_mon_vals no callback set 2026-02-25T15:00:57.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.100+0000 7fd9927fc640 1 -- 10.20.193.90:0/1993942047 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 1) ==== 939+0+0 (unknown 246333160 0 0) 0x7fd9a0005650 con 0x7fd9a4163890 2026-02-25T15:00:57.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.100+0000 7fd9927fc640 1 -- 10.20.193.90:0/1993942047 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 720+0+0 (unknown 1665658004 0 0) 0x7fd9a0006e50 con 0x7fd9a4163890 2026-02-25T15:00:57.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.100+0000 7fd9927fc640 10 client.4119.objecter ms_dispatch2 0x7fd9a4000b90 osd_map(1..1 src has 1..1) 2026-02-25T15:00:57.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.100+0000 7fd9927fc640 3 client.4119.objecter handle_osd_map got epochs [1,1] > 0 2026-02-25T15:00:57.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.100+0000 7fd9927fc640 3 client.4119.objecter handle_osd_map decoding full epoch 1 2026-02-25T15:00:57.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.100+0000 7fd9927fc640 20 client.4119.objecter dump_active .. 0 homeless 2026-02-25T15:00:57.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.129+0000 7fd9927fc640 1 -- 10.20.193.90:0/1993942047 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77519 (unknown 1092875540 0 440498749) 0x7fd9a40b3e80 con 0x7fd9a4163890 2026-02-25T15:00:57.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.129+0000 7fd9927fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:57.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.129+0000 7fd9927fc640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:00:57.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.161+0000 7fd9abb8a640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-25T15:00:57.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.161+0000 7fd9abb8a640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-25T15:00:57.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.161+0000 7fd9abb8a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:57.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.161+0000 7fd9abb8a640 1 -- 10.20.193.90:0/1993942047 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd9a4114520 con 0x7fd9a4163890 2026-02-25T15:00:57.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.209+0000 7fd9927fc640 1 -- 10.20.193.90:0/1993942047 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (unknown 2277712955 0 0) 0x7fd9a0005970 con 0x7fd9a4163890 2026-02-25T15:00:57.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.209+0000 7fd9927fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-25T15:00:57.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.209+0000 7fd9927fc640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2026-02-25T15:00:57.210 INFO:teuthology.orchestra.run.trial090.stderr:adjusted tunables profile to default 2026-02-25T15:00:57.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.209+0000 7fd9abb8a640 10 librados: watch_flush enter 2026-02-25T15:00:57.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.209+0000 7fd9abb8a640 10 librados: watch_flush exit 2026-02-25T15:00:57.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.209+0000 7fd9abb8a640 20 client.4119.objecter shutdown clearing up homeless session... 2026-02-25T15:00:57.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.209+0000 7fd9abb8a640 10 client.4119.objecter successfully canceled tick 2026-02-25T15:00:57.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.209+0000 7fd9abb8a640 10 monclient: shutdown 2026-02-25T15:00:57.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.209+0000 7fd9abb8a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:57.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.209+0000 7fd9abb8a640 1 -- 10.20.193.90:0/1993942047 >> v1:10.20.193.90:6789/0 conn(0x7fd9a4163890 legacy=0x7fd9a4113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:57.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.209+0000 7fd9abb8a640 1 -- 10.20.193.90:0/1993942047 shutdown_connections 2026-02-25T15:00:57.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.209+0000 7fd9abb8a640 1 -- 10.20.193.90:0/1993942047 >> 10.20.193.90:0/1993942047 conn(0x7fd9a40b2f10 msgr2=0x7fd9a405d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:57.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.210+0000 7fd9abb8a640 1 -- 10.20.193.90:0/1993942047 shutdown_connections 2026-02-25T15:00:57.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.210+0000 7fd9abb8a640 1 -- 10.20.193.90:0/1993942047 wait complete. 2026-02-25T15:00:57.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:00:57.210+0000 7fd9abb8a640 1 librados: shutdown 2026-02-25T15:00:57.220 INFO:tasks.ceph:check_enable_crimson: False 2026-02-25T15:00:57.220 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-25T15:00:57.220 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:57.220 DEBUG:teuthology.orchestra.run.trial090:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-25T15:00:57.246 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:57.246 DEBUG:teuthology.orchestra.run.trial090:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2026-02-25T15:00:57.311 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:57.311 DEBUG:teuthology.orchestra.run.trial090:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2026-02-25T15:00:57.376 DEBUG:teuthology.orchestra.run.trial090:> set -ex 2026-02-25T15:00:57.376 DEBUG:teuthology.orchestra.run.trial090:> sudo dd if=/var/lib/ceph/osd/ceph-12/fsid of=/dev/stdout 2026-02-25T15:00:57.439 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T15:00:57.439 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-25T15:00:57.466 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T15:00:57.466 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2026-02-25T15:00:57.531 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T15:00:57.531 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2026-02-25T15:00:57.595 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-25T15:00:57.596 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/var/lib/ceph/osd/ceph-13/fsid of=/dev/stdout 2026-02-25T15:00:57.623 INFO:tasks.ceph.mgr.x.trial189.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-25T15:00:57.623 INFO:tasks.ceph.mgr.x.trial189.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-25T15:00:57.623 INFO:tasks.ceph.mgr.x.trial189.stderr: from numpy import show_config as show_numpy_config 2026-02-25T15:00:57.660 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T15:00:57.660 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-25T15:00:57.669 INFO:tasks.ceph.mgr.y.trial090.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-25T15:00:57.669 INFO:tasks.ceph.mgr.y.trial090.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-25T15:00:57.670 INFO:tasks.ceph.mgr.y.trial090.stderr: from numpy import show_config as show_numpy_config 2026-02-25T15:00:57.685 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T15:00:57.686 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2026-02-25T15:00:57.749 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T15:00:57.749 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2026-02-25T15:00:57.813 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T15:00:57.813 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/var/lib/ceph/osd/ceph-14/fsid of=/dev/stdout 2026-02-25T15:00:57.877 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:57.877 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-25T15:00:57.901 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:57.901 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2026-02-25T15:00:57.968 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:57.968 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2026-02-25T15:00:58.030 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:00:58.030 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-15/fsid of=/dev/stdout 2026-02-25T15:00:58.094 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new ad97006e-f5c3-4e36-97a0-7fa2b250344d 0 2026-02-25T15:00:58.199 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 10 monclient: get_monmap_and_config 2026-02-25T15:00:58.199 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 10 monclient: build_initial_monmap 2026-02-25T15:00:58.200 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 10 monclient: monmap: 2026-02-25T15:00:58.200 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:58.200 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.200 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:58.198386+0000 2026-02-25T15:00:58.200 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:58.198386+0000 2026-02-25T15:00:58.200 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:58.200 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.200 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:58.200 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:58.200 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:58.201 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.201 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 1 Processor -- start 2026-02-25T15:00:58.201 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 1 -- start start 2026-02-25T15:00:58.201 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 10 monclient: init 2026-02-25T15:00:58.201 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:58.201 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:00:58.201 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 10 monclient(hunting): picked mon.noname-b con 0x7efec811e790 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:58.201 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 10 monclient(hunting): picked mon.noname-c con 0x7efec81406c0 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:58.201 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 10 monclient(hunting): picked mon.noname-a con 0x7efec8162250 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:58.201 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec8183fe0 con 0x7efec8162250 2026-02-25T15:00:58.201 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec81851e0 con 0x7efec811e790 2026-02-25T15:00:58.202 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec81863e0 con 0x7efec81406c0 2026-02-25T15:00:58.202 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:58.202 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.197+0000 7efecdda2640 10 monclient(hunting): authenticate will time out at 2965.895676s 2026-02-25T15:00:58.202 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.198+0000 7efec7fff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7efec8162250 0x7efec81826e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.189:34242/0 (socket says 10.20.193.189:34242) 2026-02-25T15:00:58.202 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.198+0000 7efec7fff640 1 -- 10.20.193.189:0/2734702692 learned_addr learned my addr 10.20.193.189:0/2734702692 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:58.208 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 1 -- 10.20.193.189:0/2734702692 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1022659061 0 0) 0x7efec81851e0 con 0x7efec811e790 2026-02-25T15:00:58.208 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.209 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.209 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.209 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 10 monclient(hunting): my global_id is 4120 2026-02-25T15:00:58.209 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 1 -- 10.20.193.189:0/2734702692 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efeac003610 con 0x7efec811e790 2026-02-25T15:00:58.209 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 1 -- 10.20.193.189:0/2734702692 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 262212926 0 0) 0x7efec81863e0 con 0x7efec81406c0 2026-02-25T15:00:58.209 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.209 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.209 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.209 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 10 monclient(hunting): my global_id is 4121 2026-02-25T15:00:58.209 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 1 -- 10.20.193.189:0/2734702692 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efec81851e0 con 0x7efec81406c0 2026-02-25T15:00:58.209 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 1 -- 10.20.193.189:0/2734702692 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3251098318 0 0) 0x7efec8183fe0 con 0x7efec8162250 2026-02-25T15:00:58.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.199+0000 7efec67fc640 10 monclient(hunting): my global_id is 4122 2026-02-25T15:00:58.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.200+0000 7efec67fc640 1 -- 10.20.193.189:0/2734702692 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efec81863e0 con 0x7efec8162250 2026-02-25T15:00:58.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efec67fc640 1 -- 10.20.193.189:0/2734702692 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 650912045 0 0) 0x7efeac003610 con 0x7efec811e790 2026-02-25T15:00:58.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efec67fc640 1 -- 10.20.193.189:0/2734702692 >> v1:10.20.193.94:6789/0 conn(0x7efec81406c0 legacy=0x7efec8160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efec67fc640 1 -- 10.20.193.189:0/2734702692 >> v1:10.20.193.90:6789/0 conn(0x7efec8162250 legacy=0x7efec81826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efec67fc640 10 monclient: _finish_hunting 0 2026-02-25T15:00:58.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efec67fc640 1 monclient: found mon.noname-b 2026-02-25T15:00:58.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efec67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:58.211 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efec67fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:00:58.211 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efec67fc640 1 -- 10.20.193.189:0/2734702692 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec8187680 con 0x7efec811e790 2026-02-25T15:00:58.211 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efec67fc640 10 monclient: _finish_auth 0 2026-02-25T15:00:58.211 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efec67fc640 10 monclient: _check_auth_tickets 2026-02-25T15:00:58.211 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efec67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:58.211 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efec67fc640 1 -- 10.20.193.189:0/2734702692 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7efeb4002df0 con 0x7efec811e790 2026-02-25T15:00:58.211 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efec67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:58.211 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efec67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:00:58.211 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efec67fc640 10 monclient: dump: 2026-02-25T15:00:58.211 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.211 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.211 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.212 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.212 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.212 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.212 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.212 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.212 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.212 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efecdda2640 5 monclient: authenticate success, global_id 4120 2026-02-25T15:00:58.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.201+0000 7efecdda2640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:00:58.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efec67fc640 1 -- 10.20.193.189:0/2734702692 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7efeb4003030 con 0x7efec811e790 2026-02-25T15:00:58.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efec67fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:58.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 10 monclient: get_monmap_and_config success 2026-02-25T15:00:58.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efec67fc640 1 -- 10.20.193.189:0/2734702692 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7efeb4003470 con 0x7efec811e790 2026-02-25T15:00:58.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 4 set_mon_vals no callback set 2026-02-25T15:00:58.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efec67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:58.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efec67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:00:58.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efec67fc640 10 monclient: dump: 2026-02-25T15:00:58.213 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.213 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.213 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.213 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.213 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.213 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.214 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.214 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.214 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.214 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 10 monclient: shutdown 2026-02-25T15:00:58.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:58.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 1 -- 10.20.193.189:0/2734702692 >> v1:10.20.193.93:6789/0 conn(0x7efec811e790 legacy=0x7efec811eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 1 -- 10.20.193.189:0/2734702692 shutdown_connections 2026-02-25T15:00:58.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 1 -- 10.20.193.189:0/2734702692 >> 10.20.193.189:0/2734702692 conn(0x7efec80588d0 msgr2=0x7efec8058cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:58.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 1 -- 10.20.193.189:0/2734702692 shutdown_connections 2026-02-25T15:00:58.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 1 -- 10.20.193.189:0/2734702692 wait complete. 2026-02-25T15:00:58.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 10 monclient: build_initial_monmap 2026-02-25T15:00:58.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 10 monclient: monmap: 2026-02-25T15:00:58.215 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:58.215 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:00:58.215 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:00:58.215 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:00:58.215 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:58.215 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:00:58.215 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:58.215 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:58.215 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:58.215 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 1 librados: starting msgr at 2026-02-25T15:00:58.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 1 librados: starting objecter 2026-02-25T15:00:58.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:00:58.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 1 Processor -- start 2026-02-25T15:00:58.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 1 -- start start 2026-02-25T15:00:58.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 1 librados: setting wanted keys 2026-02-25T15:00:58.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 1 librados: calling monclient init 2026-02-25T15:00:58.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.203+0000 7efecdda2640 10 monclient: init 2026-02-25T15:00:58.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efecdda2640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:58.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efecdda2640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:00:58.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efecdda2640 10 monclient(hunting): picked mon.noname-c con 0x7efec811e790 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:58.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efecdda2640 10 monclient(hunting): picked mon.noname-a con 0x7efec81406c0 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:58.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efecdda2640 10 monclient(hunting): picked mon.noname-b con 0x7efec8162250 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:58.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efecdda2640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec81048f0 con 0x7efec81406c0 2026-02-25T15:00:58.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efecdda2640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec810c750 con 0x7efec8162250 2026-02-25T15:00:58.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efecdda2640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec810d970 con 0x7efec811e790 2026-02-25T15:00:58.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efecdda2640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:58.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efecdda2640 10 monclient(hunting): authenticate will time out at 2965.902059s 2026-02-25T15:00:58.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efec7fff640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7efec8162250 0x7efec8161ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.189:41404/0 (socket says 10.20.193.189:41404) 2026-02-25T15:00:58.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efec7fff640 1 -- 10.20.193.189:0/2496777848 learned_addr learned my addr 10.20.193.189:0/2496777848 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:58.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efea3fff640 10 client.?.objecter ms_handle_connect 0x7efec8162250 2026-02-25T15:00:58.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efea3fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:58.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efea3fff640 10 client.?.objecter ms_handle_connect 0x7efec811e790 2026-02-25T15:00:58.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.204+0000 7efea3fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:58.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.205+0000 7efea3fff640 10 client.?.objecter ms_handle_connect 0x7efec81406c0 2026-02-25T15:00:58.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.205+0000 7efea3fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:58.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.205+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2960057013 0 0) 0x7efec810c750 con 0x7efec8162250 2026-02-25T15:00:58.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.205+0000 7efea3fff640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.205+0000 7efea3fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.205+0000 7efea3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.205+0000 7efea3fff640 10 monclient(hunting): my global_id is 4123 2026-02-25T15:00:58.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.205+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe98003610 con 0x7efec8162250 2026-02-25T15:00:58.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.205+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2245012612 0 0) 0x7efec81048f0 con 0x7efec81406c0 2026-02-25T15:00:58.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.205+0000 7efea3fff640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.205+0000 7efea3fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.205+0000 7efea3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.205+0000 7efea3fff640 10 monclient(hunting): my global_id is 4125 2026-02-25T15:00:58.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.205+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efec810c750 con 0x7efec81406c0 2026-02-25T15:00:58.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.206+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 186912869 0 0) 0x7efec810d970 con 0x7efec811e790 2026-02-25T15:00:58.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.206+0000 7efea3fff640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.206+0000 7efea3fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.206+0000 7efea3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.206+0000 7efea3fff640 10 monclient(hunting): my global_id is 4124 2026-02-25T15:00:58.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.206+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efec81048f0 con 0x7efec811e790 2026-02-25T15:00:58.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.208+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3623050715 0 0) 0x7efec810c750 con 0x7efec81406c0 2026-02-25T15:00:58.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.208+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efec810d970 con 0x7efec81406c0 2026-02-25T15:00:58.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.208+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7efeb8003070 con 0x7efec81406c0 2026-02-25T15:00:58.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.208+0000 7efea3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:58.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.208+0000 7efea3fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:00:58.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.208+0000 7efea3fff640 10 monclient(hunting): dump: 2026-02-25T15:00:58.222 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.222 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.222 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.222 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.223 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.223 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.223 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.223 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.223 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.223 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.208+0000 7efea3fff640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:00:58.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 871552385 0 0) 0x7efec810d970 con 0x7efec81406c0 2026-02-25T15:00:58.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 >> v1:10.20.193.94:6789/0 conn(0x7efec811e790 legacy=0x7efec81826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 >> v1:10.20.193.93:6789/0 conn(0x7efec8162250 legacy=0x7efec8161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efea3fff640 10 monclient: _finish_hunting 0 2026-02-25T15:00:58.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efea3fff640 1 monclient: found mon.a 2026-02-25T15:00:58.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efea3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:58.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efea3fff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:58.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec810ec10 con 0x7efec81406c0 2026-02-25T15:00:58.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efea3fff640 10 monclient: _finish_auth 0 2026-02-25T15:00:58.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efea3fff640 10 monclient: _check_auth_tickets 2026-02-25T15:00:58.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efea3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:58.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efecdda2640 5 monclient: authenticate success, global_id 4125 2026-02-25T15:00:58.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efecdda2640 10 monclient: _renew_subs 2026-02-25T15:00:58.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efecdda2640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:58.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efecdda2640 1 -- 10.20.193.189:0/2496777848 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7efec810dba0 con 0x7efec81406c0 2026-02-25T15:00:58.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efecdda2640 10 client.4125.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:00:58.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efecdda2640 10 monclient: _renew_subs 2026-02-25T15:00:58.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efecdda2640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:58.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efecdda2640 1 -- 10.20.193.189:0/2496777848 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7efec810e160 con 0x7efec81406c0 2026-02-25T15:00:58.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.209+0000 7efecdda2640 1 librados: init done 2026-02-25T15:00:58.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.210+0000 7efecdda2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:58.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.210+0000 7efecdda2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:58.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.210+0000 7efecdda2640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:58.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.210+0000 7efecdda2640 1 -- 10.20.193.189:0/2496777848 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec81846b0 con 0x7efec81406c0 2026-02-25T15:00:58.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.211+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7efeb8003700 con 0x7efec81406c0 2026-02-25T15:00:58.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.211+0000 7efea3fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:58.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.211+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7efeb8003a50 con 0x7efec81406c0 2026-02-25T15:00:58.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.211+0000 7efea3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:58.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.211+0000 7efea3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:58.226 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.211+0000 7efec5ffb640 4 set_mon_vals no callback set 2026-02-25T15:00:58.226 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.211+0000 7efea3fff640 10 monclient: dump: 2026-02-25T15:00:58.226 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.226 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.226 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.226 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.226 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.226 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.226 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.226 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.226 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.226 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.226 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.211+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 1) ==== 939+0+0 (unknown 246333160 0 0) 0x7efeb8004000 con 0x7efec81406c0 2026-02-25T15:00:58.227 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.211+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 720+0+0 (unknown 2532036530 0 0) 0x7efeb80058c0 con 0x7efec81406c0 2026-02-25T15:00:58.227 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.211+0000 7efea3fff640 10 client.4125.objecter ms_dispatch2 0x7efec8000b90 osd_map(2..2 src has 1..2) 2026-02-25T15:00:58.227 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.211+0000 7efea3fff640 3 client.4125.objecter handle_osd_map got epochs [2,2] > 0 2026-02-25T15:00:58.227 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.211+0000 7efea3fff640 3 client.4125.objecter handle_osd_map decoding full epoch 2 2026-02-25T15:00:58.227 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.211+0000 7efea3fff640 20 client.4125.objecter dump_active .. 0 homeless 2026-02-25T15:00:58.241 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.240+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77519 (unknown 1092875540 0 440498749) 0x7efeb8004320 con 0x7efec81406c0 2026-02-25T15:00:58.241 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.240+0000 7efea3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:58.242 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.240+0000 7efea3fff640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:00:58.277 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.275+0000 7efecdda2640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "ad97006e-f5c3-4e36-97a0-7fa2b250344d", "id": 0}] 2026-02-25T15:00:58.277 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.275+0000 7efecdda2640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "ad97006e-f5c3-4e36-97a0-7fa2b250344d", "id": 0}] 2026-02-25T15:00:58.277 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.275+0000 7efecdda2640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:58.277 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.275+0000 7efecdda2640 1 -- 10.20.193.189:0/2496777848 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd new", "uuid": "ad97006e-f5c3-4e36-97a0-7fa2b250344d", "id": 0} v 0) -- 0x7efec810e4a0 con 0x7efec81406c0 2026-02-25T15:00:58.322 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.321+0000 7efea3fff640 1 -- 10.20.193.189:0/2496777848 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ad97006e-f5c3-4e36-97a0-7fa2b250344d", "id": 0}]=0 v3) ==== 112+0+2 (unknown 2733756421 0 732707515) 0x7efeb8004680 con 0x7efec81406c0 2026-02-25T15:00:58.322 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.321+0000 7efea3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "ad97006e-f5c3-4e36-97a0-7fa2b250344d", "id": 0}] 2026-02-25T15:00:58.323 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.321+0000 7efea3fff640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:00:58.323 INFO:teuthology.orchestra.run.trial189.stdout:0 2026-02-25T15:00:58.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.322+0000 7efecdda2640 10 librados: watch_flush enter 2026-02-25T15:00:58.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.322+0000 7efecdda2640 10 librados: watch_flush exit 2026-02-25T15:00:58.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.322+0000 7efecdda2640 20 client.4125.objecter shutdown clearing up homeless session... 2026-02-25T15:00:58.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.322+0000 7efecdda2640 10 client.4125.objecter successfully canceled tick 2026-02-25T15:00:58.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.322+0000 7efecdda2640 10 monclient: shutdown 2026-02-25T15:00:58.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.322+0000 7efecdda2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:58.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.322+0000 7efecdda2640 1 -- 10.20.193.189:0/2496777848 >> v1:10.20.193.90:6789/0 conn(0x7efec81406c0 legacy=0x7efec8103f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.322+0000 7efecdda2640 1 -- 10.20.193.189:0/2496777848 shutdown_connections 2026-02-25T15:00:58.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.322+0000 7efecdda2640 1 -- 10.20.193.189:0/2496777848 >> 10.20.193.189:0/2496777848 conn(0x7efec80588d0 msgr2=0x7efec8058ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:58.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.322+0000 7efecdda2640 1 -- 10.20.193.189:0/2496777848 shutdown_connections 2026-02-25T15:00:58.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.322+0000 7efecdda2640 1 -- 10.20.193.189:0/2496777848 wait complete. 2026-02-25T15:00:58.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.322+0000 7efecdda2640 1 librados: shutdown 2026-02-25T15:00:58.331 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 25b02f30-96b9-43a3-9691-f06a1292e36a 1 2026-02-25T15:00:58.399 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 10 monclient: get_monmap_and_config 2026-02-25T15:00:58.399 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 10 monclient: build_initial_monmap 2026-02-25T15:00:58.399 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 10 monclient: monmap: 2026-02-25T15:00:58.399 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:58.399 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.399 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:58.398309+0000 2026-02-25T15:00:58.399 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:58.398309+0000 2026-02-25T15:00:58.399 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:58.400 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.400 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:58.400 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:58.400 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:58.400 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.400 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 1 Processor -- start 2026-02-25T15:00:58.400 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 1 -- start start 2026-02-25T15:00:58.400 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 10 monclient: init 2026-02-25T15:00:58.400 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:58.400 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:00:58.400 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 10 monclient(hunting): picked mon.noname-a con 0x7fd9dc11e810 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:58.401 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 10 monclient(hunting): picked mon.noname-b con 0x7fd9dc140740 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:58.401 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 10 monclient(hunting): picked mon.noname-c con 0x7fd9dc1622d0 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:58.401 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc184060 con 0x7fd9dc11e810 2026-02-25T15:00:58.401 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc185260 con 0x7fd9dc140740 2026-02-25T15:00:58.401 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc186460 con 0x7fd9dc1622d0 2026-02-25T15:00:58.401 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:58.401 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.397+0000 7fd9e0fed640 10 monclient(hunting): authenticate will time out at 2966.095611s 2026-02-25T15:00:58.401 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.398+0000 7fd9da575640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fd9dc11e810 0x7fd9dc11ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.189:34262/0 (socket says 10.20.193.189:34262) 2026-02-25T15:00:58.401 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.398+0000 7fd9da575640 1 -- 10.20.193.189:0/3233015084 learned_addr learned my addr 10.20.193.189:0/3233015084 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:58.407 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 1 -- 10.20.193.189:0/3233015084 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2248230038 0 0) 0x7fd9dc185260 con 0x7fd9dc140740 2026-02-25T15:00:58.408 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.408 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.408 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.408 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 10 monclient(hunting): my global_id is 4126 2026-02-25T15:00:58.408 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 1 -- 10.20.193.189:0/3233015084 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9c8003610 con 0x7fd9dc140740 2026-02-25T15:00:58.408 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 1 -- 10.20.193.189:0/3233015084 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2960961403 0 0) 0x7fd9dc186460 con 0x7fd9dc1622d0 2026-02-25T15:00:58.408 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.408 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.408 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.408 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 10 monclient(hunting): my global_id is 4127 2026-02-25T15:00:58.408 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 1 -- 10.20.193.189:0/3233015084 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9dc185260 con 0x7fd9dc1622d0 2026-02-25T15:00:58.409 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 1 -- 10.20.193.189:0/3233015084 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2911568500 0 0) 0x7fd9dc184060 con 0x7fd9dc11e810 2026-02-25T15:00:58.409 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.409 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.409 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.409 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 10 monclient(hunting): my global_id is 4128 2026-02-25T15:00:58.409 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 1 -- 10.20.193.189:0/3233015084 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9dc186460 con 0x7fd9dc11e810 2026-02-25T15:00:58.409 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9dad76640 1 -- 10.20.193.189:0/3233015084 >> v1:10.20.193.94:6789/0 conn(0x7fd9dc1622d0 legacy=0x7fd9dc182760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2026-02-25T15:00:58.409 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9dad76640 1 -- 10.20.193.189:0/3233015084 >> v1:10.20.193.94:6789/0 conn(0x7fd9dc1622d0 legacy=0x7fd9dc182760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-25T15:00:58.409 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9dad76640 1 --1- 10.20.193.189:0/3233015084 >> v1:10.20.193.94:6789/0 conn(0x7fd9dc1622d0 0x7fd9dc182760 :-1 s=OPENED pgs=19 cs=1 l=1).handle_message read tag failed 2026-02-25T15:00:58.409 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9dad76640 1 --1- 10.20.193.189:0/3233015084 >> v1:10.20.193.94:6789/0 conn(0x7fd9dc1622d0 0x7fd9dc182760 :-1 s=OPENED pgs=19 cs=1 l=1).fault on lossy channel, failing 2026-02-25T15:00:58.409 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.399+0000 7fd9d9573640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.94:6789/0 2026-02-25T15:00:58.409 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 1 -- 10.20.193.189:0/3233015084 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1613000786 0 0) 0x7fd9c8003610 con 0x7fd9dc140740 2026-02-25T15:00:58.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 1 -- 10.20.193.189:0/3233015084 >> v1:10.20.193.94:6789/0 conn(0x7fd9dc1622d0 legacy=0x7fd9dc182760 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-25T15:00:58.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 1 -- 10.20.193.189:0/3233015084 >> v1:10.20.193.90:6789/0 conn(0x7fd9dc11e810 legacy=0x7fd9dc11ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 10 monclient: _finish_hunting 0 2026-02-25T15:00:58.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 1 monclient: found mon.noname-b 2026-02-25T15:00:58.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:58.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:00:58.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 1 -- 10.20.193.189:0/3233015084 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc187700 con 0x7fd9dc140740 2026-02-25T15:00:58.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 10 monclient: _finish_auth 0 2026-02-25T15:00:58.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 10 monclient: _check_auth_tickets 2026-02-25T15:00:58.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:58.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 1 -- 10.20.193.189:0/3233015084 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd9c4002df0 con 0x7fd9dc140740 2026-02-25T15:00:58.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:58.411 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:00:58.411 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 10 monclient: dump: 2026-02-25T15:00:58.411 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.411 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.411 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.411 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.411 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.411 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.411 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.411 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.411 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.411 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.411 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9e0fed640 5 monclient: authenticate success, global_id 4126 2026-02-25T15:00:58.412 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9e0fed640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:00:58.412 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 1 -- 10.20.193.189:0/3233015084 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd9c4003030 con 0x7fd9dc140740 2026-02-25T15:00:58.412 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:58.412 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 1 -- 10.20.193.189:0/3233015084 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd9c4003430 con 0x7fd9dc140740 2026-02-25T15:00:58.412 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:58.412 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:00:58.412 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9d9573640 10 monclient: dump: 2026-02-25T15:00:58.412 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.412 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.412 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.412 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.413 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.413 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.413 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.413 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.413 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.413 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.413 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9e0fed640 10 monclient: get_monmap_and_config success 2026-02-25T15:00:58.413 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.401+0000 7fd9e0fed640 4 set_mon_vals no callback set 2026-02-25T15:00:58.413 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 10 monclient: shutdown 2026-02-25T15:00:58.413 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:58.413 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 1 -- 10.20.193.189:0/3233015084 >> v1:10.20.193.93:6789/0 conn(0x7fd9dc140740 legacy=0x7fd9dc160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.413 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 1 -- 10.20.193.189:0/3233015084 shutdown_connections 2026-02-25T15:00:58.414 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 1 -- 10.20.193.189:0/3233015084 >> 10.20.193.189:0/3233015084 conn(0x7fd9dc055dc0 msgr2=0x7fd9dc0561c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:58.414 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 1 -- 10.20.193.189:0/3233015084 shutdown_connections 2026-02-25T15:00:58.414 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 1 -- 10.20.193.189:0/3233015084 wait complete. 2026-02-25T15:00:58.414 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 10 monclient: build_initial_monmap 2026-02-25T15:00:58.414 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 10 monclient: monmap: 2026-02-25T15:00:58.414 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:58.414 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:00:58.414 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:00:58.414 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:00:58.414 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:58.414 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:00:58.414 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:58.414 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:58.415 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:58.415 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.415 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 1 librados: starting msgr at 2026-02-25T15:00:58.415 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 1 librados: starting objecter 2026-02-25T15:00:58.415 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:00:58.415 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 1 Processor -- start 2026-02-25T15:00:58.415 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 1 -- start start 2026-02-25T15:00:58.415 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 1 librados: setting wanted keys 2026-02-25T15:00:58.415 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 1 librados: calling monclient init 2026-02-25T15:00:58.415 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 10 monclient: init 2026-02-25T15:00:58.415 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:58.415 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:00:58.416 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 10 monclient(hunting): picked mon.noname-a con 0x7fd9dc11e810 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:58.416 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 10 monclient(hunting): picked mon.noname-c con 0x7fd9dc140740 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:58.416 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 10 monclient(hunting): picked mon.noname-b con 0x7fd9dc1622d0 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:58.416 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc12e7e0 con 0x7fd9dc11e810 2026-02-25T15:00:58.416 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc12f9c0 con 0x7fd9dc1622d0 2026-02-25T15:00:58.416 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc130bc0 con 0x7fd9dc140740 2026-02-25T15:00:58.416 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:58.416 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.402+0000 7fd9e0fed640 10 monclient(hunting): authenticate will time out at 2966.100840s 2026-02-25T15:00:58.416 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.403+0000 7fd9da575640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fd9dc11e810 0x7fd9dc182760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.189:34266/0 (socket says 10.20.193.189:34266) 2026-02-25T15:00:58.416 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.403+0000 7fd9da575640 1 -- 10.20.193.189:0/1875059415 learned_addr learned my addr 10.20.193.189:0/1875059415 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:58.416 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.403+0000 7fd9c2ffd640 10 client.?.objecter ms_handle_connect 0x7fd9dc11e810 2026-02-25T15:00:58.416 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.403+0000 7fd9c2ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:58.417 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.403+0000 7fd9c2ffd640 10 client.?.objecter ms_handle_connect 0x7fd9dc1622d0 2026-02-25T15:00:58.417 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.403+0000 7fd9c2ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:58.417 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.403+0000 7fd9dad76640 1 -- 10.20.193.189:0/1875059415 >> v1:10.20.193.93:6789/0 conn(0x7fd9dc1622d0 legacy=0x7fd9dc12cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-25T15:00:58.417 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.403+0000 7fd9dad76640 1 -- 10.20.193.189:0/1875059415 >> v1:10.20.193.93:6789/0 conn(0x7fd9dc1622d0 legacy=0x7fd9dc12cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-25T15:00:58.417 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.403+0000 7fd9dad76640 1 --1- 10.20.193.189:0/1875059415 >> v1:10.20.193.93:6789/0 conn(0x7fd9dc1622d0 0x7fd9dc12cf10 :-1 s=OPENED pgs=18 cs=1 l=1).handle_message read tag failed 2026-02-25T15:00:58.417 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.403+0000 7fd9dad76640 1 --1- 10.20.193.189:0/1875059415 >> v1:10.20.193.93:6789/0 conn(0x7fd9dc1622d0 0x7fd9dc12cf10 :-1 s=OPENED pgs=18 cs=1 l=1).fault on lossy channel, failing 2026-02-25T15:00:58.417 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.403+0000 7fd9c2ffd640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.93:6789/0 2026-02-25T15:00:58.417 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.403+0000 7fd9c2ffd640 10 client.?.objecter ms_handle_connect 0x7fd9dc140740 2026-02-25T15:00:58.417 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.403+0000 7fd9c2ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:58.417 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.404+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3971256092 0 0) 0x7fd9dc12e7e0 con 0x7fd9dc11e810 2026-02-25T15:00:58.417 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.404+0000 7fd9c2ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.417 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.404+0000 7fd9c2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.418 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.404+0000 7fd9c2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.418 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.404+0000 7fd9c2ffd640 10 monclient(hunting): my global_id is 4131 2026-02-25T15:00:58.418 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.404+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9a8003610 con 0x7fd9dc11e810 2026-02-25T15:00:58.418 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.404+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2235846298 0 0) 0x7fd9dc130bc0 con 0x7fd9dc140740 2026-02-25T15:00:58.418 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.404+0000 7fd9c2ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.418 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.404+0000 7fd9c2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.418 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.404+0000 7fd9c2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.418 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.404+0000 7fd9c2ffd640 10 monclient(hunting): my global_id is 4130 2026-02-25T15:00:58.418 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.404+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9dc12e7e0 con 0x7fd9dc140740 2026-02-25T15:00:58.421 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.407+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 354842167 0 0) 0x7fd9dc12e7e0 con 0x7fd9dc140740 2026-02-25T15:00:58.421 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.407+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd9dc130bc0 con 0x7fd9dc140740 2026-02-25T15:00:58.421 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.407+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd9c4002d00 con 0x7fd9dc140740 2026-02-25T15:00:58.421 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.407+0000 7fd9c2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:58.421 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.407+0000 7fd9c2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:00:58.422 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.407+0000 7fd9c2ffd640 10 monclient(hunting): dump: 2026-02-25T15:00:58.422 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.422 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.422 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.422 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.422 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.422 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.422 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.422 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.422 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.422 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.422 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.407+0000 7fd9c2ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:00:58.423 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.407+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3459378607 0 0) 0x7fd9a8003610 con 0x7fd9dc11e810 2026-02-25T15:00:58.423 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.407+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd9dc12e7e0 con 0x7fd9dc11e810 2026-02-25T15:00:58.423 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.407+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd9d0003090 con 0x7fd9dc11e810 2026-02-25T15:00:58.423 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.407+0000 7fd9c2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:58.423 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.407+0000 7fd9c2ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:58.423 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.407+0000 7fd9c2ffd640 10 monclient(hunting): dump: 2026-02-25T15:00:58.423 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.423 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.423 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.423 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.423 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.423 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.424 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.424 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.424 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.424 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.424 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2496871089 0 0) 0x7fd9dc12e7e0 con 0x7fd9dc11e810 2026-02-25T15:00:58.424 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 >> v1:10.20.193.94:6789/0 conn(0x7fd9dc140740 legacy=0x7fd9dc160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.424 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 >> v1:10.20.193.93:6789/0 conn(0x7fd9dc1622d0 legacy=0x7fd9dc12cf10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-25T15:00:58.424 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9c2ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:00:58.424 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9c2ffd640 1 monclient: found mon.a 2026-02-25T15:00:58.424 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9c2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:58.424 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9c2ffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:58.424 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc131e60 con 0x7fd9dc11e810 2026-02-25T15:00:58.425 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9c2ffd640 10 monclient: _finish_auth 0 2026-02-25T15:00:58.425 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9c2ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:00:58.425 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9c2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:58.425 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9e0fed640 5 monclient: authenticate success, global_id 4131 2026-02-25T15:00:58.425 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9e0fed640 10 monclient: _renew_subs 2026-02-25T15:00:58.425 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9e0fed640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:58.425 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9e0fed640 1 -- 10.20.193.189:0/1875059415 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd9dc12fbf0 con 0x7fd9dc11e810 2026-02-25T15:00:58.425 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9e0fed640 10 client.4131.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:00:58.425 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9e0fed640 10 monclient: _renew_subs 2026-02-25T15:00:58.425 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9e0fed640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:58.425 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9e0fed640 1 -- 10.20.193.189:0/1875059415 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd9dc130230 con 0x7fd9dc11e810 2026-02-25T15:00:58.425 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9e0fed640 1 librados: init done 2026-02-25T15:00:58.426 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9e0fed640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:58.426 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9e0fed640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:58.426 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9e0fed640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:58.426 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.409+0000 7fd9e0fed640 1 -- 10.20.193.189:0/1875059415 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9dc1869c0 con 0x7fd9dc11e810 2026-02-25T15:00:58.426 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.410+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd9d00048d0 con 0x7fd9dc11e810 2026-02-25T15:00:58.426 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.410+0000 7fd9c2ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:58.426 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.410+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd9d0004ca0 con 0x7fd9dc11e810 2026-02-25T15:00:58.426 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.410+0000 7fd9c2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:58.426 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.410+0000 7fd9c2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:58.426 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.410+0000 7fd9c2ffd640 10 monclient: dump: 2026-02-25T15:00:58.426 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.426 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.426 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.427 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.427 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.427 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.427 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.427 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.427 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.427 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.427 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.410+0000 7fd9d8d72640 4 set_mon_vals no callback set 2026-02-25T15:00:58.427 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.412+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 1) ==== 939+0+0 (unknown 246333160 0 0) 0x7fd9d0005290 con 0x7fd9dc11e810 2026-02-25T15:00:58.427 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.412+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 835+0+0 (unknown 399529944 0 0) 0x7fd9d0006ac0 con 0x7fd9dc11e810 2026-02-25T15:00:58.427 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.412+0000 7fd9c2ffd640 10 client.4131.objecter ms_dispatch2 0x7fd9dc000b90 osd_map(3..3 src has 1..3) 2026-02-25T15:00:58.427 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.412+0000 7fd9c2ffd640 3 client.4131.objecter handle_osd_map got epochs [3,3] > 0 2026-02-25T15:00:58.427 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.412+0000 7fd9c2ffd640 3 client.4131.objecter handle_osd_map decoding full epoch 3 2026-02-25T15:00:58.428 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.412+0000 7fd9c2ffd640 20 client.4131.objecter dump_active .. 0 homeless 2026-02-25T15:00:58.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.442+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77519 (unknown 1092875540 0 440498749) 0x7fd9d00055b0 con 0x7fd9dc11e810 2026-02-25T15:00:58.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.442+0000 7fd9c2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:58.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.442+0000 7fd9c2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:00:58.476 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.475+0000 7fd9e0fed640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "25b02f30-96b9-43a3-9691-f06a1292e36a", "id": 1}] 2026-02-25T15:00:58.477 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.475+0000 7fd9e0fed640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "25b02f30-96b9-43a3-9691-f06a1292e36a", "id": 1}] 2026-02-25T15:00:58.477 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.475+0000 7fd9e0fed640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:58.477 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.475+0000 7fd9e0fed640 1 -- 10.20.193.189:0/1875059415 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd new", "uuid": "25b02f30-96b9-43a3-9691-f06a1292e36a", "id": 1} v 0) -- 0x7fd9dc130520 con 0x7fd9dc11e810 2026-02-25T15:00:58.500 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.499+0000 7fd9c2ffd640 1 -- 10.20.193.189:0/1875059415 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "25b02f30-96b9-43a3-9691-f06a1292e36a", "id": 1}]=0 v4) ==== 112+0+2 (unknown 633233660 0 940483276) 0x7fd9d00059a0 con 0x7fd9dc11e810 2026-02-25T15:00:58.500 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.499+0000 7fd9c2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "25b02f30-96b9-43a3-9691-f06a1292e36a", "id": 1}] 2026-02-25T15:00:58.500 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.499+0000 7fd9c2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:00:58.500 INFO:teuthology.orchestra.run.trial189.stdout:1 2026-02-25T15:00:58.502 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.499+0000 7fd9e0fed640 10 librados: watch_flush enter 2026-02-25T15:00:58.502 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.499+0000 7fd9e0fed640 10 librados: watch_flush exit 2026-02-25T15:00:58.502 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.499+0000 7fd9e0fed640 20 client.4131.objecter shutdown clearing up homeless session... 2026-02-25T15:00:58.502 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.499+0000 7fd9e0fed640 10 client.4131.objecter successfully canceled tick 2026-02-25T15:00:58.502 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.499+0000 7fd9e0fed640 10 monclient: shutdown 2026-02-25T15:00:58.502 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.499+0000 7fd9e0fed640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:58.502 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.499+0000 7fd9e0fed640 1 -- 10.20.193.189:0/1875059415 >> v1:10.20.193.90:6789/0 conn(0x7fd9dc11e810 legacy=0x7fd9dc182760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.502 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.499+0000 7fd9e0fed640 1 -- 10.20.193.189:0/1875059415 shutdown_connections 2026-02-25T15:00:58.503 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.499+0000 7fd9e0fed640 1 -- 10.20.193.189:0/1875059415 >> 10.20.193.189:0/1875059415 conn(0x7fd9dc055dc0 msgr2=0x7fd9dc056190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:58.503 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.499+0000 7fd9e0fed640 1 -- 10.20.193.189:0/1875059415 shutdown_connections 2026-02-25T15:00:58.503 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.499+0000 7fd9e0fed640 1 -- 10.20.193.189:0/1875059415 wait complete. 2026-02-25T15:00:58.503 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.499+0000 7fd9e0fed640 1 librados: shutdown 2026-02-25T15:00:58.508 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new aeed908b-d483-498b-83ba-632e51dd05e4 2 2026-02-25T15:00:58.574 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 10 monclient: get_monmap_and_config 2026-02-25T15:00:58.574 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 10 monclient: build_initial_monmap 2026-02-25T15:00:58.574 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 10 monclient: monmap: 2026-02-25T15:00:58.574 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:58.575 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.575 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:58.573561+0000 2026-02-25T15:00:58.575 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:58.573561+0000 2026-02-25T15:00:58.575 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:58.575 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.575 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:58.575 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:58.575 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:58.575 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 1 Processor -- start 2026-02-25T15:00:58.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 1 -- start start 2026-02-25T15:00:58.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 10 monclient: init 2026-02-25T15:00:58.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:58.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:00:58.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 10 monclient(hunting): picked mon.noname-b con 0x7f569c11e790 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:58.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 10 monclient(hunting): picked mon.noname-a con 0x7f569c1406c0 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:58.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 10 monclient(hunting): picked mon.noname-c con 0x7f569c162250 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:58.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f569c183fe0 con 0x7f569c1406c0 2026-02-25T15:00:58.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f569c1851e0 con 0x7f569c11e790 2026-02-25T15:00:58.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f569c1863e0 con 0x7f569c162250 2026-02-25T15:00:58.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:58.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.572+0000 7f56a2ae1640 10 monclient(hunting): authenticate will time out at 2966.270834s 2026-02-25T15:00:58.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.573+0000 7f5693fff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f569c1406c0 0x7f569c160aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.189:34268/0 (socket says 10.20.193.189:34268) 2026-02-25T15:00:58.577 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.573+0000 7f5693fff640 1 -- 10.20.193.189:0/2706095640 learned_addr learned my addr 10.20.193.189:0/2706095640 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:58.584 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 1 -- 10.20.193.189:0/2706095640 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2115459825 0 0) 0x7f569c1851e0 con 0x7f569c11e790 2026-02-25T15:00:58.584 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.584 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.584 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.584 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 10 monclient(hunting): my global_id is 4129 2026-02-25T15:00:58.584 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 1 -- 10.20.193.189:0/2706095640 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5688003610 con 0x7f569c11e790 2026-02-25T15:00:58.584 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 1 -- 10.20.193.189:0/2706095640 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2633965495 0 0) 0x7f569c183fe0 con 0x7f569c1406c0 2026-02-25T15:00:58.584 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.584 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 10 monclient(hunting): my global_id is 4134 2026-02-25T15:00:58.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 1 -- 10.20.193.189:0/2706095640 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f569c1851e0 con 0x7f569c1406c0 2026-02-25T15:00:58.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 1 -- 10.20.193.189:0/2706095640 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2418931315 0 0) 0x7f569c1863e0 con 0x7f569c162250 2026-02-25T15:00:58.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 10 monclient(hunting): my global_id is 4133 2026-02-25T15:00:58.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.574+0000 7f56937fe640 1 -- 10.20.193.189:0/2706095640 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f569c183fe0 con 0x7f569c162250 2026-02-25T15:00:58.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56937fe640 1 -- 10.20.193.189:0/2706095640 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2874535947 0 0) 0x7f5688003610 con 0x7f569c11e790 2026-02-25T15:00:58.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56937fe640 1 -- 10.20.193.189:0/2706095640 >> v1:10.20.193.94:6789/0 conn(0x7f569c162250 legacy=0x7f569c1826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56937fe640 1 -- 10.20.193.189:0/2706095640 >> v1:10.20.193.90:6789/0 conn(0x7f569c1406c0 legacy=0x7f569c160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56937fe640 10 monclient: _finish_hunting 0 2026-02-25T15:00:58.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56937fe640 1 monclient: found mon.noname-b 2026-02-25T15:00:58.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:58.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56937fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:00:58.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56937fe640 1 -- 10.20.193.189:0/2706095640 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569c187680 con 0x7f569c11e790 2026-02-25T15:00:58.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56937fe640 10 monclient: _finish_auth 0 2026-02-25T15:00:58.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56937fe640 10 monclient: _check_auth_tickets 2026-02-25T15:00:58.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:58.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56937fe640 1 -- 10.20.193.189:0/2706095640 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f568c002df0 con 0x7f569c11e790 2026-02-25T15:00:58.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56a2ae1640 5 monclient: authenticate success, global_id 4129 2026-02-25T15:00:58.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56a2ae1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:00:58.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56937fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:58.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56937fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:00:58.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56937fe640 10 monclient: dump: 2026-02-25T15:00:58.587 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.587 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.587 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.587 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.587 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.587 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.587 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.587 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.587 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.587 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.576+0000 7f56a2ae1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:00:58.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56937fe640 1 -- 10.20.193.189:0/2706095640 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f568c003030 con 0x7f569c11e790 2026-02-25T15:00:58.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56937fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:58.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56937fe640 1 -- 10.20.193.189:0/2706095640 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f568c003470 con 0x7f569c11e790 2026-02-25T15:00:58.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 10 monclient: get_monmap_and_config success 2026-02-25T15:00:58.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 4 set_mon_vals no callback set 2026-02-25T15:00:58.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56937fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:58.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56937fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:00:58.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56937fe640 10 monclient: dump: 2026-02-25T15:00:58.588 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.588 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.588 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.588 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.589 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.589 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.589 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.589 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.589 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.589 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 10 monclient: shutdown 2026-02-25T15:00:58.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:58.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 1 -- 10.20.193.189:0/2706095640 >> v1:10.20.193.93:6789/0 conn(0x7f569c11e790 legacy=0x7f569c11eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 1 -- 10.20.193.189:0/2706095640 shutdown_connections 2026-02-25T15:00:58.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 1 -- 10.20.193.189:0/2706095640 >> 10.20.193.189:0/2706095640 conn(0x7f569c0588d0 msgr2=0x7f569c058cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:58.590 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 1 -- 10.20.193.189:0/2706095640 shutdown_connections 2026-02-25T15:00:58.590 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 1 -- 10.20.193.189:0/2706095640 wait complete. 2026-02-25T15:00:58.590 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 10 monclient: build_initial_monmap 2026-02-25T15:00:58.590 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 10 monclient: monmap: 2026-02-25T15:00:58.590 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:58.590 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:00:58.590 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:00:58.590 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:00:58.590 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:58.590 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:00:58.590 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:58.590 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:58.590 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:58.591 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 1 librados: starting msgr at 2026-02-25T15:00:58.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 1 librados: starting objecter 2026-02-25T15:00:58.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:00:58.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 1 Processor -- start 2026-02-25T15:00:58.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 1 -- start start 2026-02-25T15:00:58.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 1 librados: setting wanted keys 2026-02-25T15:00:58.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 1 librados: calling monclient init 2026-02-25T15:00:58.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 10 monclient: init 2026-02-25T15:00:58.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:58.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:00:58.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 10 monclient(hunting): picked mon.noname-a con 0x7f569c11e790 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:58.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 10 monclient(hunting): picked mon.noname-c con 0x7f569c1406c0 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:58.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 10 monclient(hunting): picked mon.noname-b con 0x7f569c162250 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:58.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f569c12e780 con 0x7f569c11e790 2026-02-25T15:00:58.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f569c12f980 con 0x7f569c162250 2026-02-25T15:00:58.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f569c130b80 con 0x7f569c1406c0 2026-02-25T15:00:58.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:58.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.577+0000 7f56a2ae1640 10 monclient(hunting): authenticate will time out at 2966.275980s 2026-02-25T15:00:58.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.578+0000 7f56a1057640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f569c162250 0x7f569c12cf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.189:41460/0 (socket says 10.20.193.189:41460) 2026-02-25T15:00:58.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.578+0000 7f56a1057640 1 -- 10.20.193.189:0/3572419349 learned_addr learned my addr 10.20.193.189:0/3572419349 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:58.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.578+0000 7f56917fa640 10 client.?.objecter ms_handle_connect 0x7f569c162250 2026-02-25T15:00:58.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.578+0000 7f56917fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:58.593 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.578+0000 7f56917fa640 10 client.?.objecter ms_handle_connect 0x7f569c1406c0 2026-02-25T15:00:58.593 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.578+0000 7f56917fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:58.593 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.578+0000 7f56917fa640 10 client.?.objecter ms_handle_connect 0x7f569c11e790 2026-02-25T15:00:58.593 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.578+0000 7f56917fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:58.593 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.579+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4253827105 0 0) 0x7f569c12f980 con 0x7f569c162250 2026-02-25T15:00:58.593 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.579+0000 7f56917fa640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.593 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.579+0000 7f56917fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.593 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.579+0000 7f56917fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.593 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.579+0000 7f56917fa640 10 monclient(hunting): my global_id is 4132 2026-02-25T15:00:58.593 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.579+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f566c003610 con 0x7f569c162250 2026-02-25T15:00:58.593 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.579+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 280824521 0 0) 0x7f569c12e780 con 0x7f569c11e790 2026-02-25T15:00:58.594 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.579+0000 7f56917fa640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.594 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.579+0000 7f56917fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.594 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.579+0000 7f56917fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.594 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.579+0000 7f56917fa640 10 monclient(hunting): my global_id is 4137 2026-02-25T15:00:58.594 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.579+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f569c12f980 con 0x7f569c11e790 2026-02-25T15:00:58.594 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.581+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1912326102 0 0) 0x7f569c130b80 con 0x7f569c1406c0 2026-02-25T15:00:58.594 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.581+0000 7f56917fa640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.594 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.581+0000 7f56917fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.594 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.581+0000 7f56917fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.594 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.581+0000 7f56917fa640 10 monclient(hunting): my global_id is 4136 2026-02-25T15:00:58.594 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.581+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f569c12e780 con 0x7f569c1406c0 2026-02-25T15:00:58.594 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.582+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4145079068 0 0) 0x7f566c003610 con 0x7f569c162250 2026-02-25T15:00:58.595 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.582+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f569c130b80 con 0x7f569c162250 2026-02-25T15:00:58.595 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.582+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5698003470 con 0x7f569c162250 2026-02-25T15:00:58.595 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.582+0000 7f56917fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:58.595 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.582+0000 7f56917fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:00:58.595 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.582+0000 7f56917fa640 10 monclient(hunting): dump: 2026-02-25T15:00:58.595 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.595 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.595 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.595 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.595 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.595 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.595 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.595 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.596 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.596 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.596 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.582+0000 7f56917fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:00:58.596 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.582+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1550799128 0 0) 0x7f569c12f980 con 0x7f569c11e790 2026-02-25T15:00:58.596 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.582+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f566c003610 con 0x7f569c11e790 2026-02-25T15:00:58.596 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.582+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f568c003380 con 0x7f569c11e790 2026-02-25T15:00:58.596 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.582+0000 7f56917fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:58.596 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.582+0000 7f56917fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:58.596 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.582+0000 7f56917fa640 10 monclient(hunting): dump: 2026-02-25T15:00:58.596 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.596 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.596 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.597 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.597 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.597 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.597 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.597 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.597 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.597 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.599 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3041967291 0 0) 0x7f566c003610 con 0x7f569c11e790 2026-02-25T15:00:58.600 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 >> v1:10.20.193.94:6789/0 conn(0x7f569c1406c0 legacy=0x7f569c160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.600 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 >> v1:10.20.193.93:6789/0 conn(0x7f569c162250 legacy=0x7f569c12cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.600 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56917fa640 10 monclient: _finish_hunting 0 2026-02-25T15:00:58.600 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56917fa640 1 monclient: found mon.a 2026-02-25T15:00:58.600 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56917fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:58.600 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56917fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:58.600 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569c131e20 con 0x7f569c11e790 2026-02-25T15:00:58.600 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56917fa640 10 monclient: _finish_auth 0 2026-02-25T15:00:58.600 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56917fa640 10 monclient: _check_auth_tickets 2026-02-25T15:00:58.600 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:58.600 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56a2ae1640 5 monclient: authenticate success, global_id 4137 2026-02-25T15:00:58.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56a2ae1640 10 monclient: _renew_subs 2026-02-25T15:00:58.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56a2ae1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:58.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56a2ae1640 1 -- 10.20.193.189:0/3572419349 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f569c130db0 con 0x7f569c11e790 2026-02-25T15:00:58.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56a2ae1640 10 client.4137.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:00:58.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56a2ae1640 10 monclient: _renew_subs 2026-02-25T15:00:58.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56a2ae1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:58.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56a2ae1640 1 -- 10.20.193.189:0/3572419349 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f569c131350 con 0x7f569c11e790 2026-02-25T15:00:58.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56a2ae1640 1 librados: init done 2026-02-25T15:00:58.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56a2ae1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:58.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56a2ae1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:58.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56a2ae1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:58.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.584+0000 7f56a2ae1640 1 -- 10.20.193.189:0/3572419349 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569c185740 con 0x7f569c11e790 2026-02-25T15:00:58.602 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.586+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f568c003520 con 0x7f569c11e790 2026-02-25T15:00:58.602 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.586+0000 7f56917fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:58.602 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.586+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f568c003a50 con 0x7f569c11e790 2026-02-25T15:00:58.602 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.586+0000 7f56917fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:58.602 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.586+0000 7f56917fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:58.602 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.586+0000 7f5692ffd640 4 set_mon_vals no callback set 2026-02-25T15:00:58.602 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.586+0000 7f56917fa640 10 monclient: dump: 2026-02-25T15:00:58.602 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.602 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.602 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.602 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.602 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.603 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.603 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.603 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.603 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.603 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.603 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.586+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 1) ==== 939+0+0 (unknown 246333160 0 0) 0x7f568c004000 con 0x7f569c11e790 2026-02-25T15:00:58.603 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.586+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 950+0+0 (unknown 2309646056 0 0) 0x7f568c005800 con 0x7f569c11e790 2026-02-25T15:00:58.603 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.586+0000 7f56917fa640 10 client.4137.objecter ms_dispatch2 0x7f569c000b90 osd_map(4..4 src has 1..4) 2026-02-25T15:00:58.603 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.586+0000 7f56917fa640 3 client.4137.objecter handle_osd_map got epochs [4,4] > 0 2026-02-25T15:00:58.603 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.586+0000 7f56917fa640 3 client.4137.objecter handle_osd_map decoding full epoch 4 2026-02-25T15:00:58.603 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.586+0000 7f56917fa640 20 client.4137.objecter dump_active .. 0 homeless 2026-02-25T15:00:58.616 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.616+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77519 (unknown 1092875540 0 440498749) 0x7f568c005510 con 0x7f569c11e790 2026-02-25T15:00:58.616 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.616+0000 7f56917fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:58.617 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.616+0000 7f56917fa640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:00:58.650 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.649+0000 7f56a2ae1640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "aeed908b-d483-498b-83ba-632e51dd05e4", "id": 2}] 2026-02-25T15:00:58.650 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.649+0000 7f56a2ae1640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "aeed908b-d483-498b-83ba-632e51dd05e4", "id": 2}] 2026-02-25T15:00:58.651 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.649+0000 7f56a2ae1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:58.651 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.649+0000 7f56a2ae1640 1 -- 10.20.193.189:0/3572419349 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd new", "uuid": "aeed908b-d483-498b-83ba-632e51dd05e4", "id": 2} v 0) -- 0x7f569c131690 con 0x7f569c11e790 2026-02-25T15:00:58.669 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.668+0000 7f56917fa640 1 -- 10.20.193.189:0/3572419349 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "aeed908b-d483-498b-83ba-632e51dd05e4", "id": 2}]=0 v5) ==== 112+0+2 (unknown 1148484193 0 216599125) 0x7f568c005a50 con 0x7f569c11e790 2026-02-25T15:00:58.669 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.668+0000 7f56917fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "aeed908b-d483-498b-83ba-632e51dd05e4", "id": 2}] 2026-02-25T15:00:58.669 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.668+0000 7f56917fa640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:00:58.669 INFO:teuthology.orchestra.run.trial189.stdout:2 2026-02-25T15:00:58.670 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.668+0000 7f56a2ae1640 10 librados: watch_flush enter 2026-02-25T15:00:58.670 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.668+0000 7f56a2ae1640 10 librados: watch_flush exit 2026-02-25T15:00:58.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.668+0000 7f56a2ae1640 20 client.4137.objecter shutdown clearing up homeless session... 2026-02-25T15:00:58.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.668+0000 7f56a2ae1640 10 client.4137.objecter successfully canceled tick 2026-02-25T15:00:58.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.668+0000 7f56a2ae1640 10 monclient: shutdown 2026-02-25T15:00:58.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.668+0000 7f56a2ae1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:58.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.668+0000 7f56a2ae1640 1 -- 10.20.193.189:0/3572419349 >> v1:10.20.193.90:6789/0 conn(0x7f569c11e790 legacy=0x7f569c1826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.668+0000 7f56a2ae1640 1 -- 10.20.193.189:0/3572419349 shutdown_connections 2026-02-25T15:00:58.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.668+0000 7f56a2ae1640 1 -- 10.20.193.189:0/3572419349 >> 10.20.193.189:0/3572419349 conn(0x7f569c0588d0 msgr2=0x7f569c1863e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:58.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.668+0000 7f56a2ae1640 1 -- 10.20.193.189:0/3572419349 shutdown_connections 2026-02-25T15:00:58.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.668+0000 7f56a2ae1640 1 -- 10.20.193.189:0/3572419349 wait complete. 2026-02-25T15:00:58.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.668+0000 7f56a2ae1640 1 librados: shutdown 2026-02-25T15:00:58.677 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new c7a30346-685f-4bd2-acaa-00b98ad3284c 3 2026-02-25T15:00:58.742 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.741+0000 7fa8c7555640 10 monclient: get_monmap_and_config 2026-02-25T15:00:58.742 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.741+0000 7fa8c7555640 10 monclient: build_initial_monmap 2026-02-25T15:00:58.742 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.741+0000 7fa8c7555640 10 monclient: monmap: 2026-02-25T15:00:58.742 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:58.742 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.743 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:58.742747+0000 2026-02-25T15:00:58.743 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:58.742747+0000 2026-02-25T15:00:58.743 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:58.743 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.743 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:58.743 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:58.743 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:58.743 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.746 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.741+0000 7fa8c7555640 1 Processor -- start 2026-02-25T15:00:58.746 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.741+0000 7fa8c7555640 1 -- start start 2026-02-25T15:00:58.746 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.741+0000 7fa8c7555640 10 monclient: init 2026-02-25T15:00:58.747 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.742+0000 7fa8c7555640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:58.747 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.742+0000 7fa8c7555640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:00:58.747 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.742+0000 7fa8c7555640 10 monclient(hunting): picked mon.noname-b con 0x7fa8c0121d90 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:58.747 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.742+0000 7fa8c7555640 10 monclient(hunting): picked mon.noname-c con 0x7fa8c0142a00 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:58.747 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.742+0000 7fa8c7555640 10 monclient(hunting): picked mon.noname-a con 0x7fa8c0164710 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:58.747 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.742+0000 7fa8c7555640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c01864c0 con 0x7fa8c0164710 2026-02-25T15:00:58.747 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.742+0000 7fa8c7555640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c01876c0 con 0x7fa8c0121d90 2026-02-25T15:00:58.747 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.742+0000 7fa8c7555640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c01888c0 con 0x7fa8c0142a00 2026-02-25T15:00:58.747 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.742+0000 7fa8c7555640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:58.747 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.742+0000 7fa8c7555640 10 monclient(hunting): authenticate will time out at 2966.440139s 2026-02-25T15:00:58.747 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.742+0000 7fa8c5acb640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fa8c0164710 0x7fa8c0184bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.189:34282/0 (socket says 10.20.193.189:34282) 2026-02-25T15:00:58.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.742+0000 7fa8c5acb640 1 -- 10.20.193.189:0/3745245716 learned_addr learned my addr 10.20.193.189:0/3745245716 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:58.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.743+0000 7fa8affff640 1 -- 10.20.193.189:0/3745245716 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3494403642 0 0) 0x7fa8c01876c0 con 0x7fa8c0121d90 2026-02-25T15:00:58.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.743+0000 7fa8affff640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.743+0000 7fa8affff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.743+0000 7fa8affff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.743+0000 7fa8affff640 10 monclient(hunting): my global_id is 4135 2026-02-25T15:00:58.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.743+0000 7fa8affff640 1 -- 10.20.193.189:0/3745245716 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8a8003610 con 0x7fa8c0121d90 2026-02-25T15:00:58.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.744+0000 7fa8affff640 1 -- 10.20.193.189:0/3745245716 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1699405320 0 0) 0x7fa8c01864c0 con 0x7fa8c0164710 2026-02-25T15:00:58.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.744+0000 7fa8affff640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.744+0000 7fa8affff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.744+0000 7fa8affff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.744+0000 7fa8affff640 10 monclient(hunting): my global_id is 4140 2026-02-25T15:00:58.749 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.744+0000 7fa8affff640 1 -- 10.20.193.189:0/3745245716 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8c01876c0 con 0x7fa8c0164710 2026-02-25T15:00:58.749 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.744+0000 7fa8affff640 1 -- 10.20.193.189:0/3745245716 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1103092892 0 0) 0x7fa8c01888c0 con 0x7fa8c0142a00 2026-02-25T15:00:58.749 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.744+0000 7fa8affff640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.749 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.744+0000 7fa8affff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.749 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.744+0000 7fa8affff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.749 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.744+0000 7fa8affff640 10 monclient(hunting): my global_id is 4139 2026-02-25T15:00:58.749 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.744+0000 7fa8affff640 1 -- 10.20.193.189:0/3745245716 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8c01864c0 con 0x7fa8c0142a00 2026-02-25T15:00:58.749 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8affff640 1 -- 10.20.193.189:0/3745245716 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3710423910 0 0) 0x7fa8a8003610 con 0x7fa8c0121d90 2026-02-25T15:00:58.749 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8affff640 1 -- 10.20.193.189:0/3745245716 >> v1:10.20.193.94:6789/0 conn(0x7fa8c0142a00 legacy=0x7fa8c0162e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.749 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8affff640 1 -- 10.20.193.189:0/3745245716 >> v1:10.20.193.90:6789/0 conn(0x7fa8c0164710 legacy=0x7fa8c0184bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.749 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8affff640 10 monclient: _finish_hunting 0 2026-02-25T15:00:58.749 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8affff640 1 monclient: found mon.noname-b 2026-02-25T15:00:58.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:58.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8affff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:00:58.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8affff640 1 -- 10.20.193.189:0/3745245716 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c0189b60 con 0x7fa8c0121d90 2026-02-25T15:00:58.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8affff640 10 monclient: _finish_auth 0 2026-02-25T15:00:58.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8affff640 10 monclient: _check_auth_tickets 2026-02-25T15:00:58.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8affff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:58.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8affff640 1 -- 10.20.193.189:0/3745245716 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa8b0002df0 con 0x7fa8c0121d90 2026-02-25T15:00:58.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8affff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:58.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8affff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:00:58.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8affff640 10 monclient: dump: 2026-02-25T15:00:58.750 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.750 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.750 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.751 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.751 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.751 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.751 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.751 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.751 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.751 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.751 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8c7555640 5 monclient: authenticate success, global_id 4135 2026-02-25T15:00:58.751 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.745+0000 7fa8c7555640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:00:58.757 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8affff640 1 -- 10.20.193.189:0/3745245716 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa8b0003030 con 0x7fa8c0121d90 2026-02-25T15:00:58.757 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8affff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:58.757 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8affff640 1 -- 10.20.193.189:0/3745245716 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa8b0003470 con 0x7fa8c0121d90 2026-02-25T15:00:58.757 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8affff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:58.757 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8affff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:00:58.757 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8affff640 10 monclient: dump: 2026-02-25T15:00:58.758 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.758 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.758 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.758 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.758 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.758 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.758 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.758 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.758 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.758 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.758 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8c7555640 10 monclient: get_monmap_and_config success 2026-02-25T15:00:58.758 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8c7555640 4 set_mon_vals no callback set 2026-02-25T15:00:58.759 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8c7555640 10 monclient: shutdown 2026-02-25T15:00:58.759 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8c7555640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:58.759 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8c7555640 1 -- 10.20.193.189:0/3745245716 >> v1:10.20.193.93:6789/0 conn(0x7fa8c0121d90 legacy=0x7fa8c0122160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.759 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8c7555640 1 -- 10.20.193.189:0/3745245716 shutdown_connections 2026-02-25T15:00:58.759 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8c7555640 1 -- 10.20.193.189:0/3745245716 >> 10.20.193.189:0/3745245716 conn(0x7fa8c0057490 msgr2=0x7fa8c0057890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:58.759 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8c7555640 1 -- 10.20.193.189:0/3745245716 shutdown_connections 2026-02-25T15:00:58.759 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8c7555640 1 -- 10.20.193.189:0/3745245716 wait complete. 2026-02-25T15:00:58.759 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8c7555640 10 monclient: build_initial_monmap 2026-02-25T15:00:58.759 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8c7555640 10 monclient: monmap: 2026-02-25T15:00:58.759 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:58.759 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:00:58.759 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:00:58.760 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:00:58.760 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:58.760 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:00:58.760 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:58.760 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:58.760 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:58.760 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.760 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8c7555640 1 librados: starting msgr at 2026-02-25T15:00:58.760 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8c7555640 1 librados: starting objecter 2026-02-25T15:00:58.760 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8c7555640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:00:58.760 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.746+0000 7fa8c7555640 1 Processor -- start 2026-02-25T15:00:58.760 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c7555640 1 -- start start 2026-02-25T15:00:58.761 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c7555640 1 librados: setting wanted keys 2026-02-25T15:00:58.761 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c7555640 1 librados: calling monclient init 2026-02-25T15:00:58.761 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c7555640 10 monclient: init 2026-02-25T15:00:58.761 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c7555640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:58.761 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c7555640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:00:58.761 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c7555640 10 monclient(hunting): picked mon.noname-c con 0x7fa8c0121d90 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:58.761 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c7555640 10 monclient(hunting): picked mon.noname-b con 0x7fa8c0142a00 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:58.761 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c7555640 10 monclient(hunting): picked mon.noname-a con 0x7fa8c0164710 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:58.761 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c7555640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c011c8e0 con 0x7fa8c0164710 2026-02-25T15:00:58.761 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c7555640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c011cab0 con 0x7fa8c0142a00 2026-02-25T15:00:58.761 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c7555640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c011cc80 con 0x7fa8c0121d90 2026-02-25T15:00:58.761 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c7555640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:58.761 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c7555640 10 monclient(hunting): authenticate will time out at 2966.445192s 2026-02-25T15:00:58.762 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c5acb640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fa8c0164710 0x7fa8c011c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.189:34286/0 (socket says 10.20.193.189:34286) 2026-02-25T15:00:58.762 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8c5acb640 1 -- 10.20.193.189:0/4016833912 learned_addr learned my addr 10.20.193.189:0/4016833912 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:58.762 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8adffb640 10 client.?.objecter ms_handle_connect 0x7fa8c0164710 2026-02-25T15:00:58.762 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8adffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:58.762 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8adffb640 10 client.?.objecter ms_handle_connect 0x7fa8c0142a00 2026-02-25T15:00:58.762 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8adffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:58.762 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8adffb640 10 client.?.objecter ms_handle_connect 0x7fa8c0121d90 2026-02-25T15:00:58.762 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.747+0000 7fa8adffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:58.762 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3036713440 0 0) 0x7fa8c011c8e0 con 0x7fa8c0164710 2026-02-25T15:00:58.762 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.762 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.762 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.763 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 10 monclient(hunting): my global_id is 4143 2026-02-25T15:00:58.763 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa890003610 con 0x7fa8c0164710 2026-02-25T15:00:58.763 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3136715505 0 0) 0x7fa8c011cab0 con 0x7fa8c0142a00 2026-02-25T15:00:58.763 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.763 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.763 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.763 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 10 monclient(hunting): my global_id is 4138 2026-02-25T15:00:58.763 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8c011c8e0 con 0x7fa8c0142a00 2026-02-25T15:00:58.763 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1968175497 0 0) 0x7fa8c011cc80 con 0x7fa8c0121d90 2026-02-25T15:00:58.763 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.763 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.763 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.764 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 10 monclient(hunting): my global_id is 4142 2026-02-25T15:00:58.764 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.748+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8c011cab0 con 0x7fa8c0121d90 2026-02-25T15:00:58.764 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.750+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1080218505 0 0) 0x7fa8c011c8e0 con 0x7fa8c0142a00 2026-02-25T15:00:58.764 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.750+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa8c011cc80 con 0x7fa8c0142a00 2026-02-25T15:00:58.764 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.750+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa8b4003090 con 0x7fa8c0142a00 2026-02-25T15:00:58.764 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.750+0000 7fa8adffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:58.764 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.750+0000 7fa8adffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:00:58.764 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.750+0000 7fa8adffb640 10 monclient(hunting): dump: 2026-02-25T15:00:58.764 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.764 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.764 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.765 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.765 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.765 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.765 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.765 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.765 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.765 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.765 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.750+0000 7fa8adffb640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:00:58.765 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.750+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2284940981 0 0) 0x7fa8c011cab0 con 0x7fa8c0121d90 2026-02-25T15:00:58.765 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.750+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa8c011c8e0 con 0x7fa8c0121d90 2026-02-25T15:00:58.765 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.750+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa8b0003380 con 0x7fa8c0121d90 2026-02-25T15:00:58.765 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.750+0000 7fa8adffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:58.766 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.750+0000 7fa8adffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:00:58.766 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.750+0000 7fa8adffb640 10 monclient(hunting): dump: 2026-02-25T15:00:58.766 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.766 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.766 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.766 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.766 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.766 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.766 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.766 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.766 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.766 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.767 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.751+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2396385482 0 0) 0x7fa890003610 con 0x7fa8c0164710 2026-02-25T15:00:58.767 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.751+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa8c011cab0 con 0x7fa8c0164710 2026-02-25T15:00:58.767 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.751+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa8bc003430 con 0x7fa8c0164710 2026-02-25T15:00:58.767 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.751+0000 7fa8adffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:58.767 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.751+0000 7fa8adffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:58.767 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.751+0000 7fa8adffb640 10 monclient(hunting): dump: 2026-02-25T15:00:58.767 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.767 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.767 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.767 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.767 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.767 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.767 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.768 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.768 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.768 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.768 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1805591295 0 0) 0x7fa8c011cc80 con 0x7fa8c0142a00 2026-02-25T15:00:58.768 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 >> v1:10.20.193.94:6789/0 conn(0x7fa8c0121d90 legacy=0x7fa8c0184bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.768 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 >> v1:10.20.193.90:6789/0 conn(0x7fa8c0164710 legacy=0x7fa8c011c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.768 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8adffb640 10 monclient: _finish_hunting 0 2026-02-25T15:00:58.768 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8adffb640 1 monclient: found mon.b 2026-02-25T15:00:58.768 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8adffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:58.768 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8adffb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:00:58.768 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c011ce50 con 0x7fa8c0142a00 2026-02-25T15:00:58.768 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8adffb640 10 monclient: _finish_auth 0 2026-02-25T15:00:58.769 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8adffb640 10 monclient: _check_auth_tickets 2026-02-25T15:00:58.769 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:58.769 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8c7555640 5 monclient: authenticate success, global_id 4138 2026-02-25T15:00:58.769 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8c7555640 10 monclient: _renew_subs 2026-02-25T15:00:58.769 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8c7555640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:00:58.769 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8c7555640 1 -- 10.20.193.189:0/4016833912 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c0150020 con 0x7fa8c0142a00 2026-02-25T15:00:58.769 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8c7555640 10 client.4138.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:00:58.769 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8c7555640 10 monclient: _renew_subs 2026-02-25T15:00:58.769 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8c7555640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:00:58.769 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8c7555640 1 -- 10.20.193.189:0/4016833912 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa8c0150610 con 0x7fa8c0142a00 2026-02-25T15:00:58.769 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.752+0000 7fa8c7555640 1 librados: init done 2026-02-25T15:00:58.769 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.753+0000 7fa8c7555640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:58.770 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.753+0000 7fa8c7555640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:58.770 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.753+0000 7fa8c7555640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:00:58.770 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.753+0000 7fa8c7555640 1 -- 10.20.193.189:0/4016833912 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c0187be0 con 0x7fa8c0142a00 2026-02-25T15:00:58.771 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.762+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa8b4003700 con 0x7fa8c0142a00 2026-02-25T15:00:58.771 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.762+0000 7fa8adffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:58.771 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.762+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa8b4003a50 con 0x7fa8c0142a00 2026-02-25T15:00:58.771 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.762+0000 7fa8adffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:58.771 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.762+0000 7fa8adffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:00:58.772 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.762+0000 7fa8af7fe640 4 set_mon_vals no callback set 2026-02-25T15:00:58.772 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.762+0000 7fa8adffb640 10 monclient: dump: 2026-02-25T15:00:58.772 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:58.772 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.772 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.772 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:58.772 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:58.772 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.772 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:58.772 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:58.772 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:58.772 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.773 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.764+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 1) ==== 939+0+0 (unknown 246333160 0 0) 0x7fa8b4004000 con 0x7fa8c0142a00 2026-02-25T15:00:58.773 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.764+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(5..5 src has 1..5) ==== 1065+0+0 (unknown 1754886420 0 0) 0x7fa8b40057e0 con 0x7fa8c0142a00 2026-02-25T15:00:58.773 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.764+0000 7fa8adffb640 10 client.4138.objecter ms_dispatch2 0x7fa8c0000b90 osd_map(5..5 src has 1..5) 2026-02-25T15:00:58.773 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.764+0000 7fa8adffb640 3 client.4138.objecter handle_osd_map got epochs [5,5] > 0 2026-02-25T15:00:58.773 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.764+0000 7fa8adffb640 3 client.4138.objecter handle_osd_map decoding full epoch 5 2026-02-25T15:00:58.773 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.764+0000 7fa8adffb640 20 client.4138.objecter dump_active .. 0 homeless 2026-02-25T15:00:58.833 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.832+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77519 (unknown 1092875540 0 440498749) 0x7fa8b4005460 con 0x7fa8c0142a00 2026-02-25T15:00:58.833 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.832+0000 7fa8adffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:58.833 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.832+0000 7fa8adffb640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:00:58.866 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.865+0000 7fa8c7555640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "c7a30346-685f-4bd2-acaa-00b98ad3284c", "id": 3}] 2026-02-25T15:00:58.866 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.865+0000 7fa8c7555640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "c7a30346-685f-4bd2-acaa-00b98ad3284c", "id": 3}] 2026-02-25T15:00:58.866 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.865+0000 7fa8c7555640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:00:58.867 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.865+0000 7fa8c7555640 1 -- 10.20.193.189:0/4016833912 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd new", "uuid": "c7a30346-685f-4bd2-acaa-00b98ad3284c", "id": 3} v 0) -- 0x7fa8c00553a0 con 0x7fa8c0142a00 2026-02-25T15:00:58.923 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.922+0000 7fa8adffb640 1 -- 10.20.193.189:0/4016833912 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c7a30346-685f-4bd2-acaa-00b98ad3284c", "id": 3}]=0 v6) ==== 112+0+2 (unknown 68951775 0 525046306) 0x7fa8c00553a0 con 0x7fa8c0142a00 2026-02-25T15:00:58.923 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.922+0000 7fa8adffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "c7a30346-685f-4bd2-acaa-00b98ad3284c", "id": 3}] 2026-02-25T15:00:58.923 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.922+0000 7fa8adffb640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:00:58.923 INFO:teuthology.orchestra.run.trial189.stdout:3 2026-02-25T15:00:58.925 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.922+0000 7fa8c7555640 10 librados: watch_flush enter 2026-02-25T15:00:58.925 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.922+0000 7fa8c7555640 10 librados: watch_flush exit 2026-02-25T15:00:58.925 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.922+0000 7fa8c7555640 20 client.4138.objecter shutdown clearing up homeless session... 2026-02-25T15:00:58.925 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.922+0000 7fa8c7555640 10 client.4138.objecter successfully canceled tick 2026-02-25T15:00:58.925 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.922+0000 7fa8c7555640 10 monclient: shutdown 2026-02-25T15:00:58.925 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.922+0000 7fa8c7555640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:58.925 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.922+0000 7fa8c7555640 1 -- 10.20.193.189:0/4016833912 >> v1:10.20.193.93:6789/0 conn(0x7fa8c0142a00 legacy=0x7fa8c0148840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:58.925 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.922+0000 7fa8c7555640 1 -- 10.20.193.189:0/4016833912 shutdown_connections 2026-02-25T15:00:58.925 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.922+0000 7fa8c7555640 1 -- 10.20.193.189:0/4016833912 >> 10.20.193.189:0/4016833912 conn(0x7fa8c0057490 msgr2=0x7fa8c01635f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:58.925 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.922+0000 7fa8c7555640 1 -- 10.20.193.189:0/4016833912 shutdown_connections 2026-02-25T15:00:58.925 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.922+0000 7fa8c7555640 1 -- 10.20.193.189:0/4016833912 wait complete. 2026-02-25T15:00:58.926 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.922+0000 7fa8c7555640 1 librados: shutdown 2026-02-25T15:00:58.931 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new d6c97301-55cc-4579-bc92-cb240c5ebe24 4 2026-02-25T15:00:58.995 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.993+0000 7f3c4295c640 10 monclient: get_monmap_and_config 2026-02-25T15:00:58.996 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.993+0000 7f3c4295c640 10 monclient: build_initial_monmap 2026-02-25T15:00:58.996 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.993+0000 7f3c4295c640 10 monclient: monmap: 2026-02-25T15:00:58.996 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:58.996 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:58.996 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:58.995000+0000 2026-02-25T15:00:58.996 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:58.995000+0000 2026-02-25T15:00:58.996 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:58.996 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:58.996 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:58.997 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:58.997 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:58.997 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:58.997 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.994+0000 7f3c4295c640 1 Processor -- start 2026-02-25T15:00:58.997 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.994+0000 7f3c4295c640 1 -- start start 2026-02-25T15:00:58.997 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.994+0000 7f3c4295c640 10 monclient: init 2026-02-25T15:00:58.997 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.994+0000 7f3c4295c640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:58.997 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.994+0000 7f3c4295c640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:00:58.997 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.994+0000 7f3c4295c640 10 monclient(hunting): picked mon.noname-c con 0x7f3c3c11fa30 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:58.997 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.994+0000 7f3c4295c640 10 monclient(hunting): picked mon.noname-b con 0x7f3c3c142940 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:58.997 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.994+0000 7f3c4295c640 10 monclient(hunting): picked mon.noname-a con 0x7f3c3c1644a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:58.997 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.994+0000 7f3c4295c640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c3c186250 con 0x7f3c3c1644a0 2026-02-25T15:00:58.998 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.994+0000 7f3c4295c640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c3c187450 con 0x7f3c3c142940 2026-02-25T15:00:58.998 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.994+0000 7f3c4295c640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c3c188650 con 0x7f3c3c11fa30 2026-02-25T15:00:58.998 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.994+0000 7f3c4295c640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:58.998 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.994+0000 7f3c4295c640 10 monclient(hunting): authenticate will time out at 2966.692312s 2026-02-25T15:00:58.998 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.994+0000 7f3c40ed2640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f3c3c1644a0 0x7f3c3c184950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.189:34300/0 (socket says 10.20.193.189:34300) 2026-02-25T15:00:58.998 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.994+0000 7f3c40ed2640 1 -- 10.20.193.189:0/4012535985 learned_addr learned my addr 10.20.193.189:0/4012535985 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:58.999 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.996+0000 7f3c3affd640 1 -- 10.20.193.189:0/4012535985 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2020401367 0 0) 0x7f3c3c186250 con 0x7f3c3c1644a0 2026-02-25T15:00:58.999 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.996+0000 7f3c3affd640 10 monclient(hunting): _negotiate 2026-02-25T15:00:58.999 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.996+0000 7f3c3affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:58.999 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.996+0000 7f3c3affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:58.999 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.996+0000 7f3c3affd640 10 monclient(hunting): my global_id is 4146 2026-02-25T15:00:58.999 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:58.996+0000 7f3c3affd640 1 -- 10.20.193.189:0/4012535985 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c20003610 con 0x7f3c3c1644a0 2026-02-25T15:00:59.003 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c3affd640 1 -- 10.20.193.189:0/4012535985 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2693922957 0 0) 0x7f3c20003610 con 0x7f3c3c1644a0 2026-02-25T15:00:59.003 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c3affd640 1 -- 10.20.193.189:0/4012535985 >> v1:10.20.193.94:6789/0 conn(0x7f3c3c11fa30 legacy=0x7f3c3c11fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.003 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c3affd640 1 -- 10.20.193.189:0/4012535985 >> v1:10.20.193.93:6789/0 conn(0x7f3c3c142940 legacy=0x7f3c3c162d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.004 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c3affd640 10 monclient: _finish_hunting 0 2026-02-25T15:00:59.004 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c3affd640 1 monclient: found mon.noname-a 2026-02-25T15:00:59.004 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c3affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:59.004 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c3affd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:00:59.004 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c3affd640 1 -- 10.20.193.189:0/4012535985 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3c1898f0 con 0x7f3c3c1644a0 2026-02-25T15:00:59.004 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c3affd640 10 monclient: _finish_auth 0 2026-02-25T15:00:59.004 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c3affd640 10 monclient: _check_auth_tickets 2026-02-25T15:00:59.004 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:59.004 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c3affd640 1 -- 10.20.193.189:0/4012535985 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3c30003170 con 0x7f3c3c1644a0 2026-02-25T15:00:59.004 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c3affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:59.004 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c3affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:00:59.005 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c3affd640 10 monclient: dump: 2026-02-25T15:00:59.005 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.005 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.005 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.005 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.005 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.005 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.005 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.005 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.005 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.005 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.005 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c4295c640 5 monclient: authenticate success, global_id 4146 2026-02-25T15:00:59.005 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.002+0000 7f3c4295c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:00:59.010 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c3affd640 1 -- 10.20.193.189:0/4012535985 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3c300033f0 con 0x7f3c3c1644a0 2026-02-25T15:00:59.010 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c3affd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:59.010 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c3affd640 1 -- 10.20.193.189:0/4012535985 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3c30003830 con 0x7f3c3c1644a0 2026-02-25T15:00:59.010 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c3affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:59.011 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c3affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:59.011 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c3affd640 10 monclient: dump: 2026-02-25T15:00:59.011 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.011 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.011 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.011 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.011 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.011 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.011 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.011 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.011 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.011 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.012 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c4295c640 10 monclient: get_monmap_and_config success 2026-02-25T15:00:59.012 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c4295c640 4 set_mon_vals no callback set 2026-02-25T15:00:59.012 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c4295c640 10 monclient: shutdown 2026-02-25T15:00:59.012 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c4295c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:59.012 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c4295c640 1 -- 10.20.193.189:0/4012535985 >> v1:10.20.193.90:6789/0 conn(0x7f3c3c1644a0 legacy=0x7f3c3c184950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.012 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c4295c640 1 -- 10.20.193.189:0/4012535985 shutdown_connections 2026-02-25T15:00:59.012 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c4295c640 1 -- 10.20.193.189:0/4012535985 >> 10.20.193.189:0/4012535985 conn(0x7f3c3c058810 msgr2=0x7f3c3c058c10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:59.012 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c4295c640 1 -- 10.20.193.189:0/4012535985 shutdown_connections 2026-02-25T15:00:59.012 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c4295c640 1 -- 10.20.193.189:0/4012535985 wait complete. 2026-02-25T15:00:59.012 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c4295c640 10 monclient: build_initial_monmap 2026-02-25T15:00:59.012 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.007+0000 7f3c4295c640 10 monclient: monmap: 2026-02-25T15:00:59.012 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:59.013 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:00:59.013 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:00:59.013 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:00:59.013 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:59.013 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:00:59.013 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:59.013 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:59.013 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:59.013 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.013 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 1 librados: starting msgr at 2026-02-25T15:00:59.013 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 1 librados: starting objecter 2026-02-25T15:00:59.013 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:00:59.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 1 Processor -- start 2026-02-25T15:00:59.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 1 -- start start 2026-02-25T15:00:59.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 1 librados: setting wanted keys 2026-02-25T15:00:59.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 1 librados: calling monclient init 2026-02-25T15:00:59.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 10 monclient: init 2026-02-25T15:00:59.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:59.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:00:59.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 10 monclient(hunting): picked mon.noname-c con 0x7f3c3c11fa30 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:59.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 10 monclient(hunting): picked mon.noname-a con 0x7f3c3c142940 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:59.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 10 monclient(hunting): picked mon.noname-b con 0x7f3c3c1644a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:59.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c3c12e930 con 0x7f3c3c142940 2026-02-25T15:00:59.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c3c12fb10 con 0x7f3c3c1644a0 2026-02-25T15:00:59.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c3c130cf0 con 0x7f3c3c11fa30 2026-02-25T15:00:59.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:59.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c4295c640 10 monclient(hunting): authenticate will time out at 2966.706336s 2026-02-25T15:00:59.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c3bfff640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f3c3c11fa30 0x7f3c3c184930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.189:60882/0 (socket says 10.20.193.189:60882) 2026-02-25T15:00:59.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c3bfff640 1 -- 10.20.193.189:0/3121220324 learned_addr learned my addr 10.20.193.189:0/3121220324 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:59.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c38ff9640 10 client.?.objecter ms_handle_connect 0x7f3c3c11fa30 2026-02-25T15:00:59.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.008+0000 7f3c38ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:59.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.009+0000 7f3c38ff9640 10 client.?.objecter ms_handle_connect 0x7f3c3c1644a0 2026-02-25T15:00:59.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.009+0000 7f3c38ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:59.067 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.066+0000 7f3c38ff9640 10 client.?.objecter ms_handle_connect 0x7f3c3c142940 2026-02-25T15:00:59.067 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.066+0000 7f3c38ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:59.069 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.068+0000 7f3c38ff9640 1 -- 10.20.193.189:0/3121220324 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3169321397 0 0) 0x7f3c3c12e930 con 0x7f3c3c142940 2026-02-25T15:00:59.069 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.068+0000 7f3c38ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:00:59.069 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.068+0000 7f3c38ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:59.069 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.068+0000 7f3c38ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:59.069 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.068+0000 7f3c38ff9640 10 monclient(hunting): my global_id is 4149 2026-02-25T15:00:59.069 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.068+0000 7f3c38ff9640 1 -- 10.20.193.189:0/3121220324 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c0c003610 con 0x7f3c3c142940 2026-02-25T15:00:59.076 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.074+0000 7f3c38ff9640 1 -- 10.20.193.189:0/3121220324 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2362620355 0 0) 0x7f3c0c003610 con 0x7f3c3c142940 2026-02-25T15:00:59.076 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.074+0000 7f3c38ff9640 1 -- 10.20.193.189:0/3121220324 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3c3c12e930 con 0x7f3c3c142940 2026-02-25T15:00:59.076 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.074+0000 7f3c38ff9640 1 -- 10.20.193.189:0/3121220324 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3c2c003090 con 0x7f3c3c142940 2026-02-25T15:00:59.076 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.074+0000 7f3c38ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:59.076 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.074+0000 7f3c38ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:00:59.076 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.074+0000 7f3c38ff9640 10 monclient(hunting): dump: 2026-02-25T15:00:59.076 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.076 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.077 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.077 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.077 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.077 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.077 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.077 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.077 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.077 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.077 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.074+0000 7f3c38ff9640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:00:59.079 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c38ff9640 1 -- 10.20.193.189:0/3121220324 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3594449537 0 0) 0x7f3c3c12e930 con 0x7f3c3c142940 2026-02-25T15:00:59.079 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c38ff9640 1 -- 10.20.193.189:0/3121220324 >> v1:10.20.193.94:6789/0 conn(0x7f3c3c11fa30 legacy=0x7f3c3c184930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.079 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c38ff9640 1 -- 10.20.193.189:0/3121220324 >> v1:10.20.193.93:6789/0 conn(0x7f3c3c1644a0 legacy=0x7f3c3c163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.079 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c38ff9640 10 monclient: _finish_hunting 0 2026-02-25T15:00:59.079 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c38ff9640 1 monclient: found mon.a 2026-02-25T15:00:59.079 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c38ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:59.079 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c38ff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:59.080 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c38ff9640 1 -- 10.20.193.189:0/3121220324 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3c131ed0 con 0x7f3c3c142940 2026-02-25T15:00:59.080 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c38ff9640 10 monclient: _finish_auth 0 2026-02-25T15:00:59.080 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c38ff9640 10 monclient: _check_auth_tickets 2026-02-25T15:00:59.080 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c38ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:59.080 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c4295c640 5 monclient: authenticate success, global_id 4149 2026-02-25T15:00:59.080 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c4295c640 10 monclient: _renew_subs 2026-02-25T15:00:59.080 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c4295c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:59.080 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c4295c640 1 -- 10.20.193.189:0/3121220324 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3c12fce0 con 0x7f3c3c142940 2026-02-25T15:00:59.080 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c4295c640 10 client.4149.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:00:59.080 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c4295c640 10 monclient: _renew_subs 2026-02-25T15:00:59.080 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c4295c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:59.080 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c4295c640 1 -- 10.20.193.189:0/3121220324 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f3c3c130320 con 0x7f3c3c142940 2026-02-25T15:00:59.081 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c4295c640 1 librados: init done 2026-02-25T15:00:59.081 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c4295c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:59.081 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c4295c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:59.081 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c4295c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:59.081 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.077+0000 7f3c4295c640 1 -- 10.20.193.189:0/3121220324 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c3c188d50 con 0x7f3c3c142940 2026-02-25T15:00:59.166 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.165+0000 7f3c38ff9640 1 -- 10.20.193.189:0/3121220324 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3c2c003440 con 0x7f3c3c142940 2026-02-25T15:00:59.166 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.165+0000 7f3c38ff9640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:59.167 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.165+0000 7f3c38ff9640 1 -- 10.20.193.189:0/3121220324 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3c2c003b90 con 0x7f3c3c142940 2026-02-25T15:00:59.167 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.165+0000 7f3c38ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:59.167 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.165+0000 7f3c38ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:59.167 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.165+0000 7f3c3a7fc640 4 set_mon_vals no callback set 2026-02-25T15:00:59.167 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.165+0000 7f3c38ff9640 10 monclient: dump: 2026-02-25T15:00:59.167 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.167 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.167 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.167 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.167 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.167 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.168 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.168 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.168 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.168 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.168 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.165+0000 7f3c38ff9640 1 -- 10.20.193.189:0/3121220324 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 2) ==== 51721+0+0 (unknown 3009573868 0 0) 0x7f3c2c003d70 con 0x7f3c3c142940 2026-02-25T15:00:59.172 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.170+0000 7f3c38ff9640 1 -- 10.20.193.189:0/3121220324 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(6..6 src has 1..6) ==== 1180+0+0 (unknown 738755471 0 0) 0x7f3c2c004090 con 0x7f3c3c142940 2026-02-25T15:00:59.172 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.170+0000 7f3c38ff9640 10 client.4149.objecter ms_dispatch2 0x7f3c3c000b90 osd_map(6..6 src has 1..6) 2026-02-25T15:00:59.172 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.170+0000 7f3c38ff9640 3 client.4149.objecter handle_osd_map got epochs [6,6] > 0 2026-02-25T15:00:59.172 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.170+0000 7f3c38ff9640 3 client.4149.objecter handle_osd_map decoding full epoch 6 2026-02-25T15:00:59.172 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.170+0000 7f3c38ff9640 20 client.4149.objecter dump_active .. 0 homeless 2026-02-25T15:00:59.172 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.171+0000 7f3c38ff9640 1 -- 10.20.193.189:0/3121220324 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77519 (unknown 1092875540 0 440498749) 0x7f3c2c0168b0 con 0x7f3c3c142940 2026-02-25T15:00:59.172 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.171+0000 7f3c38ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:59.172 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.171+0000 7f3c38ff9640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:00:59.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.204+0000 7f3c4295c640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "d6c97301-55cc-4579-bc92-cb240c5ebe24", "id": 4}] 2026-02-25T15:00:59.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.204+0000 7f3c4295c640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "d6c97301-55cc-4579-bc92-cb240c5ebe24", "id": 4}] 2026-02-25T15:00:59.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.204+0000 7f3c4295c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:59.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.204+0000 7f3c4295c640 1 -- 10.20.193.189:0/3121220324 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd new", "uuid": "d6c97301-55cc-4579-bc92-cb240c5ebe24", "id": 4} v 0) -- 0x7f3c3c0008d0 con 0x7f3c3c142940 2026-02-25T15:00:59.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.248+0000 7f3c38ff9640 1 -- 10.20.193.189:0/3121220324 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d6c97301-55cc-4579-bc92-cb240c5ebe24", "id": 4}]=0 v7) ==== 112+0+2 (unknown 1678439053 0 1697012583) 0x7f3c2c0161b0 con 0x7f3c3c142940 2026-02-25T15:00:59.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.248+0000 7f3c38ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "d6c97301-55cc-4579-bc92-cb240c5ebe24", "id": 4}] 2026-02-25T15:00:59.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.248+0000 7f3c38ff9640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:00:59.251 INFO:teuthology.orchestra.run.trial189.stdout:4 2026-02-25T15:00:59.252 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.249+0000 7f3c4295c640 10 librados: watch_flush enter 2026-02-25T15:00:59.252 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.249+0000 7f3c4295c640 10 librados: watch_flush exit 2026-02-25T15:00:59.252 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.249+0000 7f3c4295c640 20 client.4149.objecter shutdown clearing up homeless session... 2026-02-25T15:00:59.252 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.249+0000 7f3c4295c640 10 client.4149.objecter successfully canceled tick 2026-02-25T15:00:59.252 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.249+0000 7f3c4295c640 10 monclient: shutdown 2026-02-25T15:00:59.252 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.249+0000 7f3c4295c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:59.253 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.249+0000 7f3c4295c640 1 -- 10.20.193.189:0/3121220324 >> v1:10.20.193.90:6789/0 conn(0x7f3c3c142940 legacy=0x7f3c3c127100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.253 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.249+0000 7f3c4295c640 1 -- 10.20.193.189:0/3121220324 shutdown_connections 2026-02-25T15:00:59.253 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.249+0000 7f3c4295c640 1 -- 10.20.193.189:0/3121220324 >> 10.20.193.189:0/3121220324 conn(0x7f3c3c058810 msgr2=0x7f3c3c1861c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:59.253 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.249+0000 7f3c4295c640 1 -- 10.20.193.189:0/3121220324 shutdown_connections 2026-02-25T15:00:59.253 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.249+0000 7f3c4295c640 1 -- 10.20.193.189:0/3121220324 wait complete. 2026-02-25T15:00:59.253 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.249+0000 7f3c4295c640 1 librados: shutdown 2026-02-25T15:00:59.257 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new e3117058-2f0c-466b-a302-061fc4c120fe 5 2026-02-25T15:00:59.323 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.320+0000 7fab41f73640 10 monclient: get_monmap_and_config 2026-02-25T15:00:59.323 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.320+0000 7fab41f73640 10 monclient: build_initial_monmap 2026-02-25T15:00:59.323 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.320+0000 7fab41f73640 10 monclient: monmap: 2026-02-25T15:00:59.323 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:59.323 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.323 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:59.321993+0000 2026-02-25T15:00:59.323 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:59.321993+0000 2026-02-25T15:00:59.323 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:59.324 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.324 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:59.324 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:59.324 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:59.324 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab41f73640 1 Processor -- start 2026-02-25T15:00:59.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab41f73640 1 -- start start 2026-02-25T15:00:59.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab41f73640 10 monclient: init 2026-02-25T15:00:59.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab41f73640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:59.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab41f73640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:00:59.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab41f73640 10 monclient(hunting): picked mon.noname-a con 0x7fab3c11c500 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:59.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab41f73640 10 monclient(hunting): picked mon.noname-b con 0x7fab3c11e420 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:59.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab41f73640 10 monclient(hunting): picked mon.noname-c con 0x7fab3c11ffc0 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:59.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab41f73640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c120d10 con 0x7fab3c11c500 2026-02-25T15:00:59.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab41f73640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c120f00 con 0x7fab3c11e420 2026-02-25T15:00:59.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab41f73640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c1210f0 con 0x7fab3c11ffc0 2026-02-25T15:00:59.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab41f73640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:59.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab41f73640 10 monclient(hunting): authenticate will time out at 2967.019299s 2026-02-25T15:00:59.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab3affd640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fab3c11e420 0x7fab3c11e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.189:41564/0 (socket says 10.20.193.189:41564) 2026-02-25T15:00:59.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab3affd640 1 -- 10.20.193.189:0/1273150854 learned_addr learned my addr 10.20.193.189:0/1273150854 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:59.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.321+0000 7fab3bfff640 1 --1- 10.20.193.189:0/1273150854 >> v1:10.20.193.94:6789/0 conn(0x7fab3c11ffc0 0x7fab3c120420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.189:60960/0 (socket says 10.20.193.189:60960) 2026-02-25T15:00:59.346 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.344+0000 7fab3a7fc640 1 -- 10.20.193.189:0/1273150854 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2949287629 0 0) 0x7fab3c120d10 con 0x7fab3c11c500 2026-02-25T15:00:59.346 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.344+0000 7fab3a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:00:59.346 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.344+0000 7fab3a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:59.346 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.344+0000 7fab3a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:59.346 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.344+0000 7fab3a7fc640 10 monclient(hunting): my global_id is 4158 2026-02-25T15:00:59.346 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.344+0000 7fab3a7fc640 1 -- 10.20.193.189:0/1273150854 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab20003610 con 0x7fab3c11c500 2026-02-25T15:00:59.350 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.348+0000 7fab3a7fc640 1 -- 10.20.193.189:0/1273150854 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1081885323 0 0) 0x7fab20003610 con 0x7fab3c11c500 2026-02-25T15:00:59.350 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.348+0000 7fab3a7fc640 1 -- 10.20.193.189:0/1273150854 >> v1:10.20.193.94:6789/0 conn(0x7fab3c11ffc0 legacy=0x7fab3c120420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.350 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.348+0000 7fab3a7fc640 1 -- 10.20.193.189:0/1273150854 >> v1:10.20.193.93:6789/0 conn(0x7fab3c11e420 legacy=0x7fab3c11e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.350 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.348+0000 7fab3a7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:00:59.350 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.348+0000 7fab3a7fc640 1 monclient: found mon.noname-a 2026-02-25T15:00:59.350 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.348+0000 7fab3a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:59.350 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.348+0000 7fab3a7fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:00:59.351 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.348+0000 7fab3a7fc640 1 -- 10.20.193.189:0/1273150854 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3c121380 con 0x7fab3c11c500 2026-02-25T15:00:59.351 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.348+0000 7fab3a7fc640 10 monclient: _finish_auth 0 2026-02-25T15:00:59.351 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.348+0000 7fab3a7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:00:59.351 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.348+0000 7fab3a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:59.351 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.348+0000 7fab3a7fc640 1 -- 10.20.193.189:0/1273150854 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab28002df0 con 0x7fab3c11c500 2026-02-25T15:00:59.351 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.348+0000 7fab3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:59.351 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.349+0000 7fab3a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:00:59.351 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.349+0000 7fab3a7fc640 10 monclient: dump: 2026-02-25T15:00:59.351 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.351 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.351 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.351 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.352 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.352 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.352 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.352 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.352 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.352 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.352 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.349+0000 7fab41f73640 5 monclient: authenticate success, global_id 4158 2026-02-25T15:00:59.352 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.349+0000 7fab41f73640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:00:59.355 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab3a7fc640 1 -- 10.20.193.189:0/1273150854 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fab28003070 con 0x7fab3c11c500 2026-02-25T15:00:59.355 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab3a7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:59.355 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab41f73640 10 monclient: get_monmap_and_config success 2026-02-25T15:00:59.355 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab41f73640 4 set_mon_vals no callback set 2026-02-25T15:00:59.355 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab3a7fc640 1 -- 10.20.193.189:0/1273150854 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab28003470 con 0x7fab3c11c500 2026-02-25T15:00:59.355 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:59.355 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab3a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:59.356 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab3a7fc640 10 monclient: dump: 2026-02-25T15:00:59.356 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.356 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.356 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.356 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.356 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.356 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.356 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.356 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.356 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.356 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.356 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab41f73640 10 monclient: shutdown 2026-02-25T15:00:59.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab41f73640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:59.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab41f73640 1 -- 10.20.193.189:0/1273150854 >> v1:10.20.193.90:6789/0 conn(0x7fab3c11c500 legacy=0x7fab3c11c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab41f73640 1 -- 10.20.193.189:0/1273150854 shutdown_connections 2026-02-25T15:00:59.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab41f73640 1 -- 10.20.193.189:0/1273150854 >> 10.20.193.189:0/1273150854 conn(0x7fab3c0588d0 msgr2=0x7fab3c058cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:59.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab41f73640 1 -- 10.20.193.189:0/1273150854 shutdown_connections 2026-02-25T15:00:59.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab41f73640 1 -- 10.20.193.189:0/1273150854 wait complete. 2026-02-25T15:00:59.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab41f73640 10 monclient: build_initial_monmap 2026-02-25T15:00:59.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab41f73640 10 monclient: monmap: 2026-02-25T15:00:59.357 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:59.357 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:00:59.357 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:00:59.357 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:00:59.358 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:59.358 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:00:59.358 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:59.358 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:59.358 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:59.358 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.358 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab41f73640 1 librados: starting msgr at 2026-02-25T15:00:59.358 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab41f73640 1 librados: starting objecter 2026-02-25T15:00:59.358 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab41f73640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:00:59.358 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.352+0000 7fab41f73640 1 Processor -- start 2026-02-25T15:00:59.358 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab41f73640 1 -- start start 2026-02-25T15:00:59.358 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab41f73640 1 librados: setting wanted keys 2026-02-25T15:00:59.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab41f73640 1 librados: calling monclient init 2026-02-25T15:00:59.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab41f73640 10 monclient: init 2026-02-25T15:00:59.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab41f73640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:59.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab41f73640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:00:59.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab41f73640 10 monclient(hunting): picked mon.noname-b con 0x7fab3c11c500 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:59.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab41f73640 10 monclient(hunting): picked mon.noname-c con 0x7fab3c11e420 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:59.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab41f73640 10 monclient(hunting): picked mon.noname-a con 0x7fab3c11ffc0 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:59.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab41f73640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c179810 con 0x7fab3c11ffc0 2026-02-25T15:00:59.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab41f73640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c17a9f0 con 0x7fab3c11c500 2026-02-25T15:00:59.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab41f73640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c17bbd0 con 0x7fab3c11e420 2026-02-25T15:00:59.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab41f73640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:59.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab41f73640 10 monclient(hunting): authenticate will time out at 2967.051211s 2026-02-25T15:00:59.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab3bfff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fab3c11ffc0 0x7fab3c1780f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.189:34388/0 (socket says 10.20.193.189:34388) 2026-02-25T15:00:59.360 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab3bfff640 1 -- 10.20.193.189:0/1419652480 learned_addr learned my addr 10.20.193.189:0/1419652480 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:59.360 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab17fff640 10 client.?.objecter ms_handle_connect 0x7fab3c11ffc0 2026-02-25T15:00:59.360 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.353+0000 7fab17fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:59.362 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.354+0000 7fab17fff640 1 -- 10.20.193.189:0/1419652480 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2196725651 0 0) 0x7fab3c179810 con 0x7fab3c11ffc0 2026-02-25T15:00:59.362 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.354+0000 7fab17fff640 10 monclient(hunting): _negotiate 2026-02-25T15:00:59.362 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.354+0000 7fab17fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:59.362 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.354+0000 7fab17fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:59.362 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.354+0000 7fab17fff640 10 monclient(hunting): my global_id is 4161 2026-02-25T15:00:59.363 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.354+0000 7fab17fff640 1 -- 10.20.193.189:0/1419652480 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab0c003610 con 0x7fab3c11ffc0 2026-02-25T15:00:59.363 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.357+0000 7fab17fff640 1 -- 10.20.193.189:0/1419652480 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2418173738 0 0) 0x7fab0c003610 con 0x7fab3c11ffc0 2026-02-25T15:00:59.363 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.357+0000 7fab17fff640 1 -- 10.20.193.189:0/1419652480 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fab3c179810 con 0x7fab3c11ffc0 2026-02-25T15:00:59.363 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.357+0000 7fab17fff640 1 -- 10.20.193.189:0/1419652480 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab300033d0 con 0x7fab3c11ffc0 2026-02-25T15:00:59.363 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.357+0000 7fab17fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:59.363 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.357+0000 7fab17fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:00:59.363 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.357+0000 7fab17fff640 10 monclient(hunting): dump: 2026-02-25T15:00:59.363 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.363 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.364 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.364 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.364 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.364 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.364 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.364 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.364 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.364 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.364 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.357+0000 7fab17fff640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:00:59.365 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab17fff640 1 -- 10.20.193.189:0/1419652480 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 686883794 0 0) 0x7fab3c179810 con 0x7fab3c11ffc0 2026-02-25T15:00:59.365 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab17fff640 1 -- 10.20.193.189:0/1419652480 >> v1:10.20.193.94:6789/0 conn(0x7fab3c11e420 legacy=0x7fab3c1769d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.365 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab17fff640 1 -- 10.20.193.189:0/1419652480 >> v1:10.20.193.93:6789/0 conn(0x7fab3c11c500 legacy=0x7fab3c12ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.365 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab17fff640 10 monclient: _finish_hunting 0 2026-02-25T15:00:59.365 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab17fff640 1 monclient: found mon.a 2026-02-25T15:00:59.365 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab17fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:59.365 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab17fff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:59.365 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab17fff640 1 -- 10.20.193.189:0/1419652480 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3c17cdb0 con 0x7fab3c11ffc0 2026-02-25T15:00:59.365 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab17fff640 10 monclient: _finish_auth 0 2026-02-25T15:00:59.365 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab17fff640 10 monclient: _check_auth_tickets 2026-02-25T15:00:59.365 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab17fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:59.365 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab41f73640 5 monclient: authenticate success, global_id 4161 2026-02-25T15:00:59.366 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab41f73640 10 monclient: _renew_subs 2026-02-25T15:00:59.366 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab41f73640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:59.366 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab41f73640 1 -- 10.20.193.189:0/1419652480 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fab3c17bda0 con 0x7fab3c11ffc0 2026-02-25T15:00:59.366 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab41f73640 10 client.4161.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:00:59.366 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab41f73640 10 monclient: _renew_subs 2026-02-25T15:00:59.366 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab41f73640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:59.366 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab41f73640 1 -- 10.20.193.189:0/1419652480 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fab3c17c390 con 0x7fab3c11ffc0 2026-02-25T15:00:59.366 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.358+0000 7fab41f73640 1 librados: init done 2026-02-25T15:00:59.366 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.359+0000 7fab41f73640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:59.366 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.359+0000 7fab41f73640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:59.366 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.359+0000 7fab41f73640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:59.366 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.359+0000 7fab41f73640 1 -- 10.20.193.189:0/1419652480 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab3c18b5c0 con 0x7fab3c11ffc0 2026-02-25T15:00:59.368 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.362+0000 7fab17fff640 1 -- 10.20.193.189:0/1419652480 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fab30004c90 con 0x7fab3c11ffc0 2026-02-25T15:00:59.368 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.362+0000 7fab17fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:59.368 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.363+0000 7fab17fff640 1 -- 10.20.193.189:0/1419652480 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab30005060 con 0x7fab3c11ffc0 2026-02-25T15:00:59.368 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.363+0000 7fab17fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:59.368 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.363+0000 7fab17fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:59.368 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.363+0000 7fab17fff640 10 monclient: dump: 2026-02-25T15:00:59.368 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.368 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.369 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.369 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.369 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.369 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.369 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.369 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.369 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.369 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.369 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.363+0000 7fab39ffb640 4 set_mon_vals no callback set 2026-02-25T15:00:59.369 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.363+0000 7fab17fff640 1 -- 10.20.193.189:0/1419652480 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 2) ==== 51721+0+0 (unknown 3009573868 0 0) 0x7fab30005240 con 0x7fab3c11ffc0 2026-02-25T15:00:59.369 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.364+0000 7fab17fff640 1 -- 10.20.193.189:0/1419652480 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(7..7 src has 1..7) ==== 1295+0+0 (unknown 3979029745 0 0) 0x7fab30005740 con 0x7fab3c11ffc0 2026-02-25T15:00:59.369 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.364+0000 7fab17fff640 10 client.4161.objecter ms_dispatch2 0x7fab3c000b90 osd_map(7..7 src has 1..7) 2026-02-25T15:00:59.370 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.364+0000 7fab17fff640 3 client.4161.objecter handle_osd_map got epochs [7,7] > 0 2026-02-25T15:00:59.370 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.364+0000 7fab17fff640 3 client.4161.objecter handle_osd_map decoding full epoch 7 2026-02-25T15:00:59.370 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.364+0000 7fab17fff640 20 client.4161.objecter dump_active .. 0 homeless 2026-02-25T15:00:59.395 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.395+0000 7fab17fff640 1 -- 10.20.193.189:0/1419652480 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77519 (unknown 1092875540 0 440498749) 0x7fab30017df0 con 0x7fab3c11ffc0 2026-02-25T15:00:59.395 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.395+0000 7fab17fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:59.395 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.395+0000 7fab17fff640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:00:59.428 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.427+0000 7fab41f73640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "e3117058-2f0c-466b-a302-061fc4c120fe", "id": 5}] 2026-02-25T15:00:59.428 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.427+0000 7fab41f73640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "e3117058-2f0c-466b-a302-061fc4c120fe", "id": 5}] 2026-02-25T15:00:59.429 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.427+0000 7fab41f73640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:00:59.429 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.427+0000 7fab41f73640 1 -- 10.20.193.189:0/1419652480 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd new", "uuid": "e3117058-2f0c-466b-a302-061fc4c120fe", "id": 5} v 0) -- 0x7fab3c17cb30 con 0x7fab3c11ffc0 2026-02-25T15:00:59.476 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.475+0000 7fab17fff640 1 -- 10.20.193.189:0/1419652480 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e3117058-2f0c-466b-a302-061fc4c120fe", "id": 5}]=0 v8) ==== 112+0+2 (unknown 3546250721 0 1988412176) 0x7fab300176f0 con 0x7fab3c11ffc0 2026-02-25T15:00:59.476 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.475+0000 7fab17fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "e3117058-2f0c-466b-a302-061fc4c120fe", "id": 5}] 2026-02-25T15:00:59.476 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.475+0000 7fab17fff640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:00:59.477 INFO:teuthology.orchestra.run.trial189.stdout:5 2026-02-25T15:00:59.478 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.475+0000 7fab41f73640 10 librados: watch_flush enter 2026-02-25T15:00:59.478 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.475+0000 7fab41f73640 10 librados: watch_flush exit 2026-02-25T15:00:59.478 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.475+0000 7fab41f73640 20 client.4161.objecter shutdown clearing up homeless session... 2026-02-25T15:00:59.478 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.475+0000 7fab41f73640 10 client.4161.objecter successfully canceled tick 2026-02-25T15:00:59.478 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.475+0000 7fab41f73640 10 monclient: shutdown 2026-02-25T15:00:59.479 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.475+0000 7fab41f73640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:59.479 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.475+0000 7fab41f73640 1 -- 10.20.193.189:0/1419652480 >> v1:10.20.193.90:6789/0 conn(0x7fab3c11ffc0 legacy=0x7fab3c1780f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.479 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.475+0000 7fab41f73640 1 -- 10.20.193.189:0/1419652480 shutdown_connections 2026-02-25T15:00:59.479 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.475+0000 7fab41f73640 1 -- 10.20.193.189:0/1419652480 >> 10.20.193.189:0/1419652480 conn(0x7fab3c0588d0 msgr2=0x7fab3c059040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:59.479 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.475+0000 7fab41f73640 1 -- 10.20.193.189:0/1419652480 shutdown_connections 2026-02-25T15:00:59.479 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.475+0000 7fab41f73640 1 -- 10.20.193.189:0/1419652480 wait complete. 2026-02-25T15:00:59.479 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.475+0000 7fab41f73640 1 librados: shutdown 2026-02-25T15:00:59.485 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new f4b4a3ce-0a0b-4386-a433-dcbf65558ecf 6 2026-02-25T15:00:59.550 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.547+0000 7f39fadb1640 10 monclient: get_monmap_and_config 2026-02-25T15:00:59.550 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.547+0000 7f39fadb1640 10 monclient: build_initial_monmap 2026-02-25T15:00:59.550 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.547+0000 7f39fadb1640 10 monclient: monmap: 2026-02-25T15:00:59.550 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:59.551 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.551 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:59.549244+0000 2026-02-25T15:00:59.551 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:59.549244+0000 2026-02-25T15:00:59.551 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:59.551 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.551 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:59.551 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:59.551 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:59.551 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.551 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.548+0000 7f39fadb1640 1 Processor -- start 2026-02-25T15:00:59.551 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.548+0000 7f39fadb1640 1 -- start start 2026-02-25T15:00:59.552 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.548+0000 7f39fadb1640 10 monclient: init 2026-02-25T15:00:59.552 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.548+0000 7f39fadb1640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:59.552 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.548+0000 7f39fadb1640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:00:59.552 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.548+0000 7f39fadb1640 10 monclient(hunting): picked mon.noname-b con 0x7f39f4121db0 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:59.552 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.548+0000 7f39fadb1640 10 monclient(hunting): picked mon.noname-a con 0x7f39f4142a00 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:59.552 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.548+0000 7f39fadb1640 10 monclient(hunting): picked mon.noname-c con 0x7f39f4164710 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:59.552 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.548+0000 7f39fadb1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39f41864c0 con 0x7f39f4142a00 2026-02-25T15:00:59.552 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.548+0000 7f39fadb1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39f41876c0 con 0x7f39f4121db0 2026-02-25T15:00:59.552 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.548+0000 7f39fadb1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39f41888c0 con 0x7f39f4164710 2026-02-25T15:00:59.552 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.548+0000 7f39fadb1640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:59.552 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.548+0000 7f39fadb1640 10 monclient(hunting): authenticate will time out at 2967.246629s 2026-02-25T15:00:59.552 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.548+0000 7f39f8b26640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f39f4121db0 0x7f39f4122180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.189:41592/0 (socket says 10.20.193.189:41592) 2026-02-25T15:00:59.553 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.548+0000 7f39f8b26640 1 -- 10.20.193.189:0/3550990818 learned_addr learned my addr 10.20.193.189:0/3550990818 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:59.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 1 -- 10.20.193.189:0/3550990818 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1500863703 0 0) 0x7f39f41888c0 con 0x7f39f4164710 2026-02-25T15:00:59.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:00:59.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:59.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:59.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 10 monclient(hunting): my global_id is 4169 2026-02-25T15:00:59.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 1 -- 10.20.193.189:0/3550990818 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39e0003610 con 0x7f39f4164710 2026-02-25T15:00:59.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 1 -- 10.20.193.189:0/3550990818 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3265893067 0 0) 0x7f39f41864c0 con 0x7f39f4142a00 2026-02-25T15:00:59.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:00:59.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:59.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:59.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 10 monclient(hunting): my global_id is 4164 2026-02-25T15:00:59.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 1 -- 10.20.193.189:0/3550990818 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39f41888c0 con 0x7f39f4142a00 2026-02-25T15:00:59.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 1 -- 10.20.193.189:0/3550990818 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1319685657 0 0) 0x7f39f41876c0 con 0x7f39f4121db0 2026-02-25T15:00:59.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:00:59.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:59.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:59.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 10 monclient(hunting): my global_id is 4162 2026-02-25T15:00:59.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.550+0000 7f39eb7fe640 1 -- 10.20.193.189:0/3550990818 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39f41864c0 con 0x7f39f4121db0 2026-02-25T15:00:59.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.551+0000 7f39eb7fe640 1 -- 10.20.193.189:0/3550990818 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2150295538 0 0) 0x7f39e0003610 con 0x7f39f4164710 2026-02-25T15:00:59.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.551+0000 7f39eb7fe640 1 -- 10.20.193.189:0/3550990818 >> v1:10.20.193.93:6789/0 conn(0x7f39f4121db0 legacy=0x7f39f4122180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.551+0000 7f39eb7fe640 1 -- 10.20.193.189:0/3550990818 >> v1:10.20.193.90:6789/0 conn(0x7f39f4142a00 legacy=0x7f39f4162e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.551+0000 7f39eb7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:00:59.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.551+0000 7f39eb7fe640 1 monclient: found mon.noname-c 2026-02-25T15:00:59.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.551+0000 7f39eb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:59.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.551+0000 7f39eb7fe640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:00:59.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.551+0000 7f39eb7fe640 1 -- 10.20.193.189:0/3550990818 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f4189b60 con 0x7f39f4164710 2026-02-25T15:00:59.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.551+0000 7f39eb7fe640 10 monclient: _finish_auth 0 2026-02-25T15:00:59.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.551+0000 7f39eb7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:00:59.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.551+0000 7f39eb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:59.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.551+0000 7f39eb7fe640 1 -- 10.20.193.189:0/3550990818 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f39f0003170 con 0x7f39f4164710 2026-02-25T15:00:59.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.551+0000 7f39eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:59.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.552+0000 7f39eb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:00:59.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.552+0000 7f39eb7fe640 10 monclient: dump: 2026-02-25T15:00:59.564 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.564 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.564 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.564 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.564 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.564 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.564 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.564 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.564 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.564 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.564 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.552+0000 7f39fadb1640 5 monclient: authenticate success, global_id 4169 2026-02-25T15:00:59.564 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.552+0000 7f39fadb1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:00:59.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39eb7fe640 1 -- 10.20.193.189:0/3550990818 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f39f00033b0 con 0x7f39f4164710 2026-02-25T15:00:59.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39eb7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:59.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39eb7fe640 1 -- 10.20.193.189:0/3550990818 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f39f00037f0 con 0x7f39f4164710 2026-02-25T15:00:59.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 10 monclient: get_monmap_and_config success 2026-02-25T15:00:59.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:59.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 4 set_mon_vals no callback set 2026-02-25T15:00:59.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39eb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:00:59.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39eb7fe640 10 monclient: dump: 2026-02-25T15:00:59.565 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.565 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.565 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.565 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.566 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.566 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.566 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.566 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.566 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.566 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.566 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 10 monclient: shutdown 2026-02-25T15:00:59.566 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:59.566 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 1 -- 10.20.193.189:0/3550990818 >> v1:10.20.193.94:6789/0 conn(0x7f39f4164710 legacy=0x7f39f4184bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.566 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 1 -- 10.20.193.189:0/3550990818 shutdown_connections 2026-02-25T15:00:59.566 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 1 -- 10.20.193.189:0/3550990818 >> 10.20.193.189:0/3550990818 conn(0x7f39f4057440 msgr2=0x7f39f4057840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:59.566 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 1 -- 10.20.193.189:0/3550990818 shutdown_connections 2026-02-25T15:00:59.566 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 1 -- 10.20.193.189:0/3550990818 wait complete. 2026-02-25T15:00:59.567 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 10 monclient: build_initial_monmap 2026-02-25T15:00:59.567 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 10 monclient: monmap: 2026-02-25T15:00:59.567 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:59.567 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:00:59.567 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:00:59.567 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:00:59.567 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:59.567 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:00:59.567 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:59.567 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:59.567 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:59.567 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 1 librados: starting msgr at 2026-02-25T15:00:59.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 1 librados: starting objecter 2026-02-25T15:00:59.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:00:59.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 1 Processor -- start 2026-02-25T15:00:59.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 1 -- start start 2026-02-25T15:00:59.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 1 librados: setting wanted keys 2026-02-25T15:00:59.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 1 librados: calling monclient init 2026-02-25T15:00:59.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.553+0000 7f39fadb1640 10 monclient: init 2026-02-25T15:00:59.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39fadb1640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:59.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39fadb1640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:00:59.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39fadb1640 10 monclient(hunting): picked mon.noname-a con 0x7f39f4121db0 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:59.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39fadb1640 10 monclient(hunting): picked mon.noname-c con 0x7f39f4142a00 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:59.569 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39fadb1640 10 monclient(hunting): picked mon.noname-b con 0x7f39f4164710 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:59.569 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39fadb1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39f411c8e0 con 0x7f39f4121db0 2026-02-25T15:00:59.569 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39fadb1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39f411cab0 con 0x7f39f4164710 2026-02-25T15:00:59.569 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39fadb1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39f411cc80 con 0x7f39f4142a00 2026-02-25T15:00:59.569 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39fadb1640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:59.569 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39fadb1640 10 monclient(hunting): authenticate will time out at 2967.252078s 2026-02-25T15:00:59.569 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39f8b26640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f39f4121db0 0x7f39f4162e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.189:34398/0 (socket says 10.20.193.189:34398) 2026-02-25T15:00:59.569 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39f8b26640 1 -- 10.20.193.189:0/3400620410 learned_addr learned my addr 10.20.193.189:0/3400620410 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:59.569 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39e97fa640 10 client.?.objecter ms_handle_connect 0x7f39f4121db0 2026-02-25T15:00:59.569 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39e97fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:59.569 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39e97fa640 10 client.?.objecter ms_handle_connect 0x7f39f4164710 2026-02-25T15:00:59.569 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39e97fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:59.570 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39e97fa640 10 client.?.objecter ms_handle_connect 0x7f39f4142a00 2026-02-25T15:00:59.570 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.554+0000 7f39e97fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:59.570 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3928344951 0 0) 0x7f39f411cc80 con 0x7f39f4142a00 2026-02-25T15:00:59.570 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 10 monclient(hunting): _negotiate 2026-02-25T15:00:59.570 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:59.570 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:59.570 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 10 monclient(hunting): my global_id is 4172 2026-02-25T15:00:59.570 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39c4003610 con 0x7f39f4142a00 2026-02-25T15:00:59.570 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1676438572 0 0) 0x7f39f411cab0 con 0x7f39f4164710 2026-02-25T15:00:59.570 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 10 monclient(hunting): _negotiate 2026-02-25T15:00:59.570 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:59.570 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:59.570 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 10 monclient(hunting): my global_id is 4165 2026-02-25T15:00:59.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39f411cc80 con 0x7f39f4164710 2026-02-25T15:00:59.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1746269986 0 0) 0x7f39f411c8e0 con 0x7f39f4121db0 2026-02-25T15:00:59.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 10 monclient(hunting): _negotiate 2026-02-25T15:00:59.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:59.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:59.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 10 monclient(hunting): my global_id is 4167 2026-02-25T15:00:59.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.555+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39f411cab0 con 0x7f39f4121db0 2026-02-25T15:00:59.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.557+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 353310947 0 0) 0x7f39c4003610 con 0x7f39f4142a00 2026-02-25T15:00:59.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.557+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f39f411c8e0 con 0x7f39f4142a00 2026-02-25T15:00:59.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.557+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f39dc003040 con 0x7f39f4142a00 2026-02-25T15:00:59.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.557+0000 7f39e97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:59.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.557+0000 7f39e97fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:00:59.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.557+0000 7f39e97fa640 10 monclient(hunting): dump: 2026-02-25T15:00:59.572 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.572 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.572 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.572 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.572 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.572 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.572 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.572 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.572 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.572 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.557+0000 7f39e97fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:00:59.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.557+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 34143565 0 0) 0x7f39f411cc80 con 0x7f39f4164710 2026-02-25T15:00:59.573 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.557+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f39c4003610 con 0x7f39f4164710 2026-02-25T15:00:59.573 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.557+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f39f0003560 con 0x7f39f4164710 2026-02-25T15:00:59.573 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.557+0000 7f39e97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:59.573 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.557+0000 7f39e97fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:00:59.573 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.557+0000 7f39e97fa640 10 monclient(hunting): dump: 2026-02-25T15:00:59.573 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.573 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.573 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.573 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.573 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.573 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.573 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.574 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.574 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.574 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.574 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.558+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 735560734 0 0) 0x7f39f411cab0 con 0x7f39f4121db0 2026-02-25T15:00:59.574 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.558+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f39f411cc80 con 0x7f39f4121db0 2026-02-25T15:00:59.574 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.558+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f39e4003000 con 0x7f39f4121db0 2026-02-25T15:00:59.574 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.558+0000 7f39e97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:59.574 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.558+0000 7f39e97fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:59.574 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.558+0000 7f39e97fa640 10 monclient(hunting): dump: 2026-02-25T15:00:59.574 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.574 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.574 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.575 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.575 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.575 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.575 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.575 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.575 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.575 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2774605505 0 0) 0x7f39f411c8e0 con 0x7f39f4142a00 2026-02-25T15:00:59.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 >> v1:10.20.193.93:6789/0 conn(0x7f39f4164710 legacy=0x7f39f411c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 >> v1:10.20.193.90:6789/0 conn(0x7f39f4121db0 legacy=0x7f39f4162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39e97fa640 10 monclient: _finish_hunting 0 2026-02-25T15:00:59.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39e97fa640 1 monclient: found mon.c 2026-02-25T15:00:59.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39e97fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:59.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39e97fa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:00:59.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f411ce50 con 0x7f39f4142a00 2026-02-25T15:00:59.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39e97fa640 10 monclient: _finish_auth 0 2026-02-25T15:00:59.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39e97fa640 10 monclient: _check_auth_tickets 2026-02-25T15:00:59.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39e97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:59.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39fadb1640 5 monclient: authenticate success, global_id 4172 2026-02-25T15:00:59.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39fadb1640 10 monclient: _renew_subs 2026-02-25T15:00:59.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39fadb1640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:00:59.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39fadb1640 1 -- 10.20.193.189:0/3400620410 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f39f4171df0 con 0x7f39f4142a00 2026-02-25T15:00:59.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39fadb1640 10 client.4172.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:00:59.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39fadb1640 10 monclient: _renew_subs 2026-02-25T15:00:59.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39fadb1640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:00:59.576 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39fadb1640 1 -- 10.20.193.189:0/3400620410 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f39f41723e0 con 0x7f39f4142a00 2026-02-25T15:00:59.577 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39fadb1640 1 librados: init done 2026-02-25T15:00:59.577 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39fadb1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:59.577 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39fadb1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:59.577 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39fadb1640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:00:59.577 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.559+0000 7f39fadb1640 1 -- 10.20.193.189:0/3400620410 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f4187be0 con 0x7f39f4142a00 2026-02-25T15:00:59.578 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.573+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f39dc0031e0 con 0x7f39f4142a00 2026-02-25T15:00:59.578 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.573+0000 7f39e97fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:59.578 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.573+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f39dc0038e0 con 0x7f39f4142a00 2026-02-25T15:00:59.579 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.573+0000 7f39e97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:59.579 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.573+0000 7f39e97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:00:59.579 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.573+0000 7f39e97fa640 10 monclient: dump: 2026-02-25T15:00:59.579 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.579 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.579 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.579 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.579 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.579 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.579 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.579 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.579 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.580 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.580 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.573+0000 7f39eaffd640 4 set_mon_vals no callback set 2026-02-25T15:00:59.580 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.574+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 2) ==== 51721+0+0 (unknown 3009573868 0 0) 0x7f39dc0104e0 con 0x7f39f4142a00 2026-02-25T15:00:59.580 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.576+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(8..8 src has 1..8) ==== 1410+0+0 (unknown 555142716 0 0) 0x7f39dc0157b0 con 0x7f39f4142a00 2026-02-25T15:00:59.580 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.576+0000 7f39e97fa640 10 client.4172.objecter ms_dispatch2 0x7f39f4000b90 osd_map(8..8 src has 1..8) 2026-02-25T15:00:59.580 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.576+0000 7f39e97fa640 3 client.4172.objecter handle_osd_map got epochs [8,8] > 0 2026-02-25T15:00:59.580 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.576+0000 7f39e97fa640 3 client.4172.objecter handle_osd_map decoding full epoch 8 2026-02-25T15:00:59.580 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.576+0000 7f39e97fa640 20 client.4172.objecter dump_active .. 0 homeless 2026-02-25T15:00:59.644 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.643+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77519 (unknown 1092875540 0 440498749) 0x7f39dc016590 con 0x7f39f4142a00 2026-02-25T15:00:59.644 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.643+0000 7f39e97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:59.644 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.643+0000 7f39e97fa640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:00:59.678 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.677+0000 7f39fadb1640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "f4b4a3ce-0a0b-4386-a433-dcbf65558ecf", "id": 6}] 2026-02-25T15:00:59.678 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.677+0000 7f39fadb1640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "f4b4a3ce-0a0b-4386-a433-dcbf65558ecf", "id": 6}] 2026-02-25T15:00:59.678 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.677+0000 7f39fadb1640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:00:59.678 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.677+0000 7f39fadb1640 1 -- 10.20.193.189:0/3400620410 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd new", "uuid": "f4b4a3ce-0a0b-4386-a433-dcbf65558ecf", "id": 6} v 0) -- 0x7f39f40553a0 con 0x7f39f4142a00 2026-02-25T15:00:59.753 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.752+0000 7f39e97fa640 1 -- 10.20.193.189:0/3400620410 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f4b4a3ce-0a0b-4386-a433-dcbf65558ecf", "id": 6}]=0 v9) ==== 112+0+2 (unknown 1916250567 0 1113811849) 0x7f39f40553a0 con 0x7f39f4142a00 2026-02-25T15:00:59.753 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.752+0000 7f39e97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "f4b4a3ce-0a0b-4386-a433-dcbf65558ecf", "id": 6}] 2026-02-25T15:00:59.753 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.752+0000 7f39e97fa640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:00:59.754 INFO:teuthology.orchestra.run.trial189.stdout:6 2026-02-25T15:00:59.755 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.752+0000 7f39fadb1640 10 librados: watch_flush enter 2026-02-25T15:00:59.755 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.752+0000 7f39fadb1640 10 librados: watch_flush exit 2026-02-25T15:00:59.755 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.752+0000 7f39fadb1640 20 client.4172.objecter shutdown clearing up homeless session... 2026-02-25T15:00:59.755 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.752+0000 7f39fadb1640 10 client.4172.objecter successfully canceled tick 2026-02-25T15:00:59.755 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.752+0000 7f39fadb1640 10 monclient: shutdown 2026-02-25T15:00:59.756 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.752+0000 7f39fadb1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:59.756 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.752+0000 7f39fadb1640 1 -- 10.20.193.189:0/3400620410 >> v1:10.20.193.94:6789/0 conn(0x7f39f4142a00 legacy=0x7f39f416a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.756 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.753+0000 7f39fadb1640 1 -- 10.20.193.189:0/3400620410 shutdown_connections 2026-02-25T15:00:59.756 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.753+0000 7f39fadb1640 1 -- 10.20.193.189:0/3400620410 >> 10.20.193.189:0/3400620410 conn(0x7f39f4057440 msgr2=0x7f39f411f820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:59.756 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.753+0000 7f39fadb1640 1 -- 10.20.193.189:0/3400620410 shutdown_connections 2026-02-25T15:00:59.756 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.753+0000 7f39fadb1640 1 -- 10.20.193.189:0/3400620410 wait complete. 2026-02-25T15:00:59.756 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.753+0000 7f39fadb1640 1 librados: shutdown 2026-02-25T15:00:59.760 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 3337e549-6b6e-444e-997b-18a19beb5640 7 2026-02-25T15:00:59.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.823+0000 7ff98d0ce640 10 monclient: get_monmap_and_config 2026-02-25T15:00:59.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.823+0000 7ff98d0ce640 10 monclient: build_initial_monmap 2026-02-25T15:00:59.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.823+0000 7ff98d0ce640 10 monclient: monmap: 2026-02-25T15:00:59.825 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:59.825 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.825 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:59.825084+0000 2026-02-25T15:00:59.825 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:59.825084+0000 2026-02-25T15:00:59.825 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:59.825 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.825 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:59.825 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:59.825 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:59.826 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.824+0000 7ff98d0ce640 1 Processor -- start 2026-02-25T15:00:59.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.824+0000 7ff98d0ce640 1 -- start start 2026-02-25T15:00:59.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.824+0000 7ff98d0ce640 10 monclient: init 2026-02-25T15:00:59.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.824+0000 7ff98d0ce640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:59.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.824+0000 7ff98d0ce640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:00:59.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.824+0000 7ff98d0ce640 10 monclient(hunting): picked mon.noname-c con 0x7ff98805d9b0 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:59.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.824+0000 7ff98d0ce640 10 monclient(hunting): picked mon.noname-b con 0x7ff98805dd80 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:59.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.824+0000 7ff98d0ce640 10 monclient(hunting): picked mon.noname-a con 0x7ff9881649b0 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:59.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.824+0000 7ff98d0ce640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff98805e200 con 0x7ff9881649b0 2026-02-25T15:00:59.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.824+0000 7ff98d0ce640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9881874c0 con 0x7ff98805dd80 2026-02-25T15:00:59.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.824+0000 7ff98d0ce640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9881886a0 con 0x7ff98805d9b0 2026-02-25T15:00:59.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.824+0000 7ff98d0ce640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:59.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.824+0000 7ff98d0ce640 10 monclient(hunting): authenticate will time out at 2967.522435s 2026-02-25T15:00:59.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.824+0000 7ff986575640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7ff98805dd80 0x7ff98811b8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.189:41612/0 (socket says 10.20.193.189:41612) 2026-02-25T15:00:59.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.824+0000 7ff986575640 1 -- 10.20.193.189:0/2535212834 learned_addr learned my addr 10.20.193.189:0/2535212834 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:59.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 1 -- 10.20.193.189:0/2535212834 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 354297590 0 0) 0x7ff9881874c0 con 0x7ff98805dd80 2026-02-25T15:00:59.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 10 monclient(hunting): _negotiate 2026-02-25T15:00:59.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:59.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:59.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 10 monclient(hunting): my global_id is 4168 2026-02-25T15:00:59.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 1 -- 10.20.193.189:0/2535212834 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff974003610 con 0x7ff98805dd80 2026-02-25T15:00:59.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 1 -- 10.20.193.189:0/2535212834 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2979899251 0 0) 0x7ff98805e200 con 0x7ff9881649b0 2026-02-25T15:00:59.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 10 monclient(hunting): _negotiate 2026-02-25T15:00:59.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:59.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:59.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 10 monclient(hunting): my global_id is 4170 2026-02-25T15:00:59.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 1 -- 10.20.193.189:0/2535212834 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9881874c0 con 0x7ff9881649b0 2026-02-25T15:00:59.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 1 -- 10.20.193.189:0/2535212834 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2631177071 0 0) 0x7ff9881886a0 con 0x7ff98805d9b0 2026-02-25T15:00:59.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 10 monclient(hunting): _negotiate 2026-02-25T15:00:59.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:59.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:59.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 10 monclient(hunting): my global_id is 4175 2026-02-25T15:00:59.831 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.826+0000 7ff985d74640 1 -- 10.20.193.189:0/2535212834 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff98805e200 con 0x7ff98805d9b0 2026-02-25T15:00:59.837 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff985d74640 1 -- 10.20.193.189:0/2535212834 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1014077463 0 0) 0x7ff974003610 con 0x7ff98805dd80 2026-02-25T15:00:59.837 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff985d74640 1 -- 10.20.193.189:0/2535212834 >> v1:10.20.193.94:6789/0 conn(0x7ff98805d9b0 legacy=0x7ff98811b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.837 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff985d74640 1 -- 10.20.193.189:0/2535212834 >> v1:10.20.193.90:6789/0 conn(0x7ff9881649b0 legacy=0x7ff988184d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.837 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff985d74640 10 monclient: _finish_hunting 0 2026-02-25T15:00:59.837 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff985d74640 1 monclient: found mon.noname-b 2026-02-25T15:00:59.837 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff985d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:59.837 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff985d74640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:00:59.838 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff985d74640 1 -- 10.20.193.189:0/2535212834 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff988189920 con 0x7ff98805dd80 2026-02-25T15:00:59.838 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff985d74640 10 monclient: _finish_auth 0 2026-02-25T15:00:59.838 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff985d74640 10 monclient: _check_auth_tickets 2026-02-25T15:00:59.838 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff985d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:59.838 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff985d74640 1 -- 10.20.193.189:0/2535212834 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff970002df0 con 0x7ff98805dd80 2026-02-25T15:00:59.838 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff985d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:59.838 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff985d74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:00:59.838 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff985d74640 10 monclient: dump: 2026-02-25T15:00:59.838 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.838 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.838 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.838 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.839 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.839 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.839 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.839 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.839 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.839 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.839 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff98d0ce640 5 monclient: authenticate success, global_id 4168 2026-02-25T15:00:59.839 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.827+0000 7ff98d0ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:00:59.839 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.828+0000 7ff985d74640 1 -- 10.20.193.189:0/2535212834 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff970003030 con 0x7ff98805dd80 2026-02-25T15:00:59.839 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.828+0000 7ff985d74640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:59.839 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.828+0000 7ff985d74640 1 -- 10.20.193.189:0/2535212834 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff970003470 con 0x7ff98805dd80 2026-02-25T15:00:59.839 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.828+0000 7ff985d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:59.840 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.828+0000 7ff985d74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:00:59.840 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.828+0000 7ff985d74640 10 monclient: dump: 2026-02-25T15:00:59.840 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.840 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.840 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.840 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.840 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.840 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.840 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.840 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.840 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.840 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.828+0000 7ff98d0ce640 10 monclient: get_monmap_and_config success 2026-02-25T15:00:59.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.828+0000 7ff98d0ce640 4 set_mon_vals no callback set 2026-02-25T15:00:59.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 10 monclient: shutdown 2026-02-25T15:00:59.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:59.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 1 -- 10.20.193.189:0/2535212834 >> v1:10.20.193.93:6789/0 conn(0x7ff98805dd80 legacy=0x7ff98811b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 1 -- 10.20.193.189:0/2535212834 shutdown_connections 2026-02-25T15:00:59.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 1 -- 10.20.193.189:0/2535212834 >> 10.20.193.189:0/2535212834 conn(0x7ff9880588d0 msgr2=0x7ff988058cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:59.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 1 -- 10.20.193.189:0/2535212834 shutdown_connections 2026-02-25T15:00:59.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 1 -- 10.20.193.189:0/2535212834 wait complete. 2026-02-25T15:00:59.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 10 monclient: build_initial_monmap 2026-02-25T15:00:59.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 10 monclient: monmap: 2026-02-25T15:00:59.842 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:00:59.842 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:00:59.842 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:00:59.842 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:00:59.842 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:00:59.842 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:00:59.842 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:00:59.842 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:00:59.842 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:00:59.842 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.842 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 1 librados: starting msgr at 2026-02-25T15:00:59.842 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 1 librados: starting objecter 2026-02-25T15:00:59.843 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:00:59.843 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 1 Processor -- start 2026-02-25T15:00:59.843 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 1 -- start start 2026-02-25T15:00:59.843 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 1 librados: setting wanted keys 2026-02-25T15:00:59.843 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 1 librados: calling monclient init 2026-02-25T15:00:59.843 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 10 monclient: init 2026-02-25T15:00:59.843 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 10 monclient: _reopen_session rank -1 2026-02-25T15:00:59.843 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:00:59.843 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 10 monclient(hunting): picked mon.noname-a con 0x7ff98805d9b0 addr v1:10.20.193.90:6789/0 2026-02-25T15:00:59.843 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 10 monclient(hunting): picked mon.noname-b con 0x7ff98805dd80 addr v1:10.20.193.93:6789/0 2026-02-25T15:00:59.843 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 10 monclient(hunting): picked mon.noname-c con 0x7ff9881649b0 addr v1:10.20.193.94:6789/0 2026-02-25T15:00:59.843 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff988102340 con 0x7ff98805d9b0 2026-02-25T15:00:59.843 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff988109220 con 0x7ff98805dd80 2026-02-25T15:00:59.844 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff98810a400 con 0x7ff9881649b0 2026-02-25T15:00:59.844 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 10 monclient(hunting): _renew_subs 2026-02-25T15:00:59.844 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.829+0000 7ff98d0ce640 10 monclient(hunting): authenticate will time out at 2967.527840s 2026-02-25T15:00:59.844 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.830+0000 7ff986575640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7ff98805dd80 0x7ff988161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.189:41624/0 (socket says 10.20.193.189:41624) 2026-02-25T15:00:59.844 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.830+0000 7ff986575640 1 -- 10.20.193.189:0/2673900623 learned_addr learned my addr 10.20.193.189:0/2673900623 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:00:59.844 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.830+0000 7ff9677fe640 10 client.?.objecter ms_handle_connect 0x7ff98805dd80 2026-02-25T15:00:59.844 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.830+0000 7ff9677fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:59.844 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.830+0000 7ff9677fe640 10 client.?.objecter ms_handle_connect 0x7ff9881649b0 2026-02-25T15:00:59.844 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.830+0000 7ff9677fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:59.844 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.831+0000 7ff9677fe640 10 client.?.objecter ms_handle_connect 0x7ff98805d9b0 2026-02-25T15:00:59.844 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.831+0000 7ff9677fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:00:59.844 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.831+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1075592096 0 0) 0x7ff988109220 con 0x7ff98805dd80 2026-02-25T15:00:59.845 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.831+0000 7ff9677fe640 10 monclient(hunting): _negotiate 2026-02-25T15:00:59.845 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.831+0000 7ff9677fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:59.845 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.831+0000 7ff9677fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:59.845 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.831+0000 7ff9677fe640 10 monclient(hunting): my global_id is 4171 2026-02-25T15:00:59.845 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.831+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff954003610 con 0x7ff98805dd80 2026-02-25T15:00:59.845 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.832+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1777074594 0 0) 0x7ff98810a400 con 0x7ff9881649b0 2026-02-25T15:00:59.845 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.832+0000 7ff9677fe640 10 monclient(hunting): _negotiate 2026-02-25T15:00:59.845 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.832+0000 7ff9677fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:59.845 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.832+0000 7ff9677fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:59.845 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.832+0000 7ff9677fe640 10 monclient(hunting): my global_id is 4178 2026-02-25T15:00:59.845 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.832+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff988109220 con 0x7ff9881649b0 2026-02-25T15:00:59.845 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.832+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4058988029 0 0) 0x7ff988102340 con 0x7ff98805d9b0 2026-02-25T15:00:59.845 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.832+0000 7ff9677fe640 10 monclient(hunting): _negotiate 2026-02-25T15:00:59.846 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.832+0000 7ff9677fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:00:59.846 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.832+0000 7ff9677fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:00:59.846 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.832+0000 7ff9677fe640 10 monclient(hunting): my global_id is 4173 2026-02-25T15:00:59.846 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.832+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff98810a400 con 0x7ff98805d9b0 2026-02-25T15:00:59.846 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.833+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3144553986 0 0) 0x7ff954003610 con 0x7ff98805dd80 2026-02-25T15:00:59.846 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.833+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff988102340 con 0x7ff98805dd80 2026-02-25T15:00:59.846 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.833+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff970003380 con 0x7ff98805dd80 2026-02-25T15:00:59.846 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.833+0000 7ff9677fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:59.846 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.833+0000 7ff9677fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:00:59.846 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.833+0000 7ff9677fe640 10 monclient(hunting): dump: 2026-02-25T15:00:59.846 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.846 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.846 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.847 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.847 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.847 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.847 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.847 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.847 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.847 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.847 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.833+0000 7ff9677fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:00:59.847 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.834+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 991505898 0 0) 0x7ff988109220 con 0x7ff9881649b0 2026-02-25T15:00:59.847 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.834+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff954003610 con 0x7ff9881649b0 2026-02-25T15:00:59.847 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.834+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff978003400 con 0x7ff9881649b0 2026-02-25T15:00:59.847 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.834+0000 7ff9677fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:59.848 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.834+0000 7ff9677fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:00:59.848 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.834+0000 7ff9677fe640 10 monclient(hunting): dump: 2026-02-25T15:00:59.848 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.848 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.848 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.848 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.848 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.848 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.848 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.848 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.848 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.848 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.849 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.834+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3311540197 0 0) 0x7ff98810a400 con 0x7ff98805d9b0 2026-02-25T15:00:59.849 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.834+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff988109220 con 0x7ff98805d9b0 2026-02-25T15:00:59.849 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.834+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff97c003060 con 0x7ff98805d9b0 2026-02-25T15:00:59.849 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.834+0000 7ff9677fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:00:59.849 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.834+0000 7ff9677fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:00:59.849 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.834+0000 7ff9677fe640 10 monclient(hunting): dump: 2026-02-25T15:00:59.849 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.849 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.849 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.849 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.849 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.849 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.849 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.850 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.850 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.850 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.850 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4108862425 0 0) 0x7ff988102340 con 0x7ff98805dd80 2026-02-25T15:00:59.850 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 >> v1:10.20.193.94:6789/0 conn(0x7ff9881649b0 legacy=0x7ff9881630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.850 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 >> v1:10.20.193.90:6789/0 conn(0x7ff98805d9b0 legacy=0x7ff988101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.850 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff9677fe640 10 monclient: _finish_hunting 0 2026-02-25T15:00:59.850 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff9677fe640 1 monclient: found mon.b 2026-02-25T15:00:59.850 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff9677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:00:59.850 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff9677fe640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:00:59.850 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98810b5e0 con 0x7ff98805dd80 2026-02-25T15:00:59.850 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff9677fe640 10 monclient: _finish_auth 0 2026-02-25T15:00:59.850 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff9677fe640 10 monclient: _check_auth_tickets 2026-02-25T15:00:59.851 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff9677fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:00:59.851 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff98d0ce640 5 monclient: authenticate success, global_id 4171 2026-02-25T15:00:59.851 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff98d0ce640 10 monclient: _renew_subs 2026-02-25T15:00:59.851 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff98d0ce640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:00:59.851 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff98d0ce640 1 -- 10.20.193.189:0/2673900623 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff98810a5d0 con 0x7ff98805dd80 2026-02-25T15:00:59.851 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff98d0ce640 10 client.4171.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:00:59.851 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff98d0ce640 10 monclient: _renew_subs 2026-02-25T15:00:59.851 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff98d0ce640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:00:59.851 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff98d0ce640 1 -- 10.20.193.189:0/2673900623 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff98810ab70 con 0x7ff98805dd80 2026-02-25T15:00:59.851 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff98d0ce640 1 librados: init done 2026-02-25T15:00:59.851 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff98d0ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:59.851 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff98d0ce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:59.851 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff98d0ce640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:00:59.852 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.835+0000 7ff98d0ce640 1 -- 10.20.193.189:0/2673900623 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9881869b0 con 0x7ff98805dd80 2026-02-25T15:00:59.853 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.839+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff970002f80 con 0x7ff98805dd80 2026-02-25T15:00:59.853 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.839+0000 7ff9677fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:00:59.853 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.839+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff970004ce0 con 0x7ff98805dd80 2026-02-25T15:00:59.853 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.839+0000 7ff9677fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:00:59.853 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.839+0000 7ff9677fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:00:59.853 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.839+0000 7ff9677fe640 10 monclient: dump: 2026-02-25T15:00:59.853 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:00:59.853 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:00:59.853 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.854 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:00:59.854 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:00:59.854 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:00:59.854 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:00:59.854 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:00:59.854 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:00:59.854 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:00:59.854 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.839+0000 7ff985573640 4 set_mon_vals no callback set 2026-02-25T15:00:59.854 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.840+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 2) ==== 51721+0+0 (unknown 3009573868 0 0) 0x7ff9700118a0 con 0x7ff98805dd80 2026-02-25T15:00:59.854 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.841+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(9..9 src has 1..9) ==== 1525+0+0 (unknown 2301162597 0 0) 0x7ff970015020 con 0x7ff98805dd80 2026-02-25T15:00:59.854 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.841+0000 7ff9677fe640 10 client.4171.objecter ms_dispatch2 0x7ff988000b90 osd_map(9..9 src has 1..9) 2026-02-25T15:00:59.854 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.841+0000 7ff9677fe640 3 client.4171.objecter handle_osd_map got epochs [9,9] > 0 2026-02-25T15:00:59.854 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.841+0000 7ff9677fe640 3 client.4171.objecter handle_osd_map decoding full epoch 9 2026-02-25T15:00:59.854 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.841+0000 7ff9677fe640 20 client.4171.objecter dump_active .. 0 homeless 2026-02-25T15:00:59.869 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.869+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77519 (unknown 1092875540 0 440498749) 0x7ff970017800 con 0x7ff98805dd80 2026-02-25T15:00:59.870 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.869+0000 7ff9677fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:00:59.870 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.869+0000 7ff9677fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:00:59.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.901+0000 7ff98d0ce640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "3337e549-6b6e-444e-997b-18a19beb5640", "id": 7}] 2026-02-25T15:00:59.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.901+0000 7ff98d0ce640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "3337e549-6b6e-444e-997b-18a19beb5640", "id": 7}] 2026-02-25T15:00:59.903 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.901+0000 7ff98d0ce640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:00:59.903 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.901+0000 7ff98d0ce640 1 -- 10.20.193.189:0/2673900623 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd new", "uuid": "3337e549-6b6e-444e-997b-18a19beb5640", "id": 7} v 0) -- 0x7ff98810b2b0 con 0x7ff98805dd80 2026-02-25T15:00:59.939 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.938+0000 7ff9677fe640 1 -- 10.20.193.189:0/2673900623 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3337e549-6b6e-444e-997b-18a19beb5640", "id": 7}]=0 v10) ==== 112+0+2 (unknown 3089056933 0 1371665406) 0x7ff970017100 con 0x7ff98805dd80 2026-02-25T15:00:59.939 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.938+0000 7ff9677fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "3337e549-6b6e-444e-997b-18a19beb5640", "id": 7}] 2026-02-25T15:00:59.939 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.938+0000 7ff9677fe640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:00:59.939 INFO:teuthology.orchestra.run.trial189.stdout:7 2026-02-25T15:00:59.941 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.938+0000 7ff98d0ce640 10 librados: watch_flush enter 2026-02-25T15:00:59.941 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.938+0000 7ff98d0ce640 10 librados: watch_flush exit 2026-02-25T15:00:59.941 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.938+0000 7ff98d0ce640 20 client.4171.objecter shutdown clearing up homeless session... 2026-02-25T15:00:59.941 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.938+0000 7ff98d0ce640 10 client.4171.objecter successfully canceled tick 2026-02-25T15:00:59.941 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.938+0000 7ff98d0ce640 10 monclient: shutdown 2026-02-25T15:00:59.941 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.938+0000 7ff98d0ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:00:59.941 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.938+0000 7ff98d0ce640 1 -- 10.20.193.189:0/2673900623 >> v1:10.20.193.93:6789/0 conn(0x7ff98805dd80 legacy=0x7ff988161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:00:59.941 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.938+0000 7ff98d0ce640 1 -- 10.20.193.189:0/2673900623 shutdown_connections 2026-02-25T15:00:59.941 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.938+0000 7ff98d0ce640 1 -- 10.20.193.189:0/2673900623 >> 10.20.193.189:0/2673900623 conn(0x7ff9880588d0 msgr2=0x7ff98805c050 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:00:59.941 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.938+0000 7ff98d0ce640 1 -- 10.20.193.189:0/2673900623 shutdown_connections 2026-02-25T15:00:59.942 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.938+0000 7ff98d0ce640 1 -- 10.20.193.189:0/2673900623 wait complete. 2026-02-25T15:00:59.942 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:00:59.938+0000 7ff98d0ce640 1 librados: shutdown 2026-02-25T15:00:59.947 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new b292f7c4-9a41-4e48-876c-29281c5ec73b 8 2026-02-25T15:01:00.013 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.010+0000 7f477e49c640 10 monclient: get_monmap_and_config 2026-02-25T15:01:00.013 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.010+0000 7f477e49c640 10 monclient: build_initial_monmap 2026-02-25T15:01:00.013 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.010+0000 7f477e49c640 10 monclient: monmap: 2026-02-25T15:01:00.013 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:00.013 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.013 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:01:00.012041+0000 2026-02-25T15:01:00.013 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:01:00.012041+0000 2026-02-25T15:01:00.013 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:00.013 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.014 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:00.014 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:00.014 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:00.014 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.011+0000 7f477e49c640 1 Processor -- start 2026-02-25T15:01:00.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.011+0000 7f477e49c640 1 -- start start 2026-02-25T15:01:00.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.011+0000 7f477e49c640 10 monclient: init 2026-02-25T15:01:00.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.011+0000 7f477e49c640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:00.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.011+0000 7f477e49c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:00.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.011+0000 7f477e49c640 10 monclient(hunting): picked mon.noname-a con 0x7f477811e790 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:00.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.011+0000 7f477e49c640 10 monclient(hunting): picked mon.noname-b con 0x7f47781406c0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:00.014 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.011+0000 7f477e49c640 10 monclient(hunting): picked mon.noname-c con 0x7f4778162250 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:00.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.011+0000 7f477e49c640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4778183fe0 con 0x7f477811e790 2026-02-25T15:01:00.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.011+0000 7f477e49c640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47781851e0 con 0x7f47781406c0 2026-02-25T15:01:00.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.011+0000 7f477e49c640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47781863e0 con 0x7f4778162250 2026-02-25T15:01:00.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.011+0000 7f477e49c640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:00.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.011+0000 7f477e49c640 10 monclient(hunting): authenticate will time out at 2967.709504s 2026-02-25T15:01:00.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.011+0000 7f47777fe640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f47781406c0 0x7f4778160aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.189:41630/0 (socket says 10.20.193.189:41630) 2026-02-25T15:01:00.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.011+0000 7f47777fe640 1 -- 10.20.193.189:0/3146813947 learned_addr learned my addr 10.20.193.189:0/3146813947 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:00.020 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.013+0000 7f4776ffd640 1 -- 10.20.193.189:0/3146813947 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2375041589 0 0) 0x7f47781851e0 con 0x7f47781406c0 2026-02-25T15:01:00.020 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.013+0000 7f4776ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.020 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.013+0000 7f4776ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.020 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.013+0000 7f4776ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.020 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.013+0000 7f4776ffd640 10 monclient(hunting): my global_id is 4174 2026-02-25T15:01:00.020 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.013+0000 7f4776ffd640 1 -- 10.20.193.189:0/3146813947 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f475c003610 con 0x7f47781406c0 2026-02-25T15:01:00.020 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.013+0000 7f4776ffd640 1 -- 10.20.193.189:0/3146813947 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3370485272 0 0) 0x7f47781863e0 con 0x7f4778162250 2026-02-25T15:01:00.020 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.013+0000 7f4776ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.020 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.013+0000 7f4776ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.020 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.013+0000 7f4776ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.020 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.013+0000 7f4776ffd640 10 monclient(hunting): my global_id is 4181 2026-02-25T15:01:00.020 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.013+0000 7f4776ffd640 1 -- 10.20.193.189:0/3146813947 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47781851e0 con 0x7f4778162250 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f4776ffd640 1 -- 10.20.193.189:0/3146813947 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2278357498 0 0) 0x7f475c003610 con 0x7f47781406c0 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f4776ffd640 1 -- 10.20.193.189:0/3146813947 >> v1:10.20.193.94:6789/0 conn(0x7f4778162250 legacy=0x7f47781826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f4776ffd640 1 -- 10.20.193.189:0/3146813947 >> v1:10.20.193.90:6789/0 conn(0x7f477811e790 legacy=0x7f477811eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f4776ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f4776ffd640 1 monclient: found mon.noname-b 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f4776ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f4776ffd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f4776ffd640 1 -- 10.20.193.189:0/3146813947 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4778187680 con 0x7f47781406c0 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f4776ffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f4776ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f4776ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f4776ffd640 1 -- 10.20.193.189:0/3146813947 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4768002df0 con 0x7f47781406c0 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f4776ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f4776ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f4776ffd640 10 monclient: dump: 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f477e49c640 5 monclient: authenticate success, global_id 4174 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.014+0000 7f477e49c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.015+0000 7f4776ffd640 1 -- 10.20.193.189:0/3146813947 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4768003030 con 0x7f47781406c0 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.015+0000 7f4776ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.015+0000 7f4776ffd640 1 -- 10.20.193.189:0/3146813947 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4768003470 con 0x7f47781406c0 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.015+0000 7f4776ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.015+0000 7f4776ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.015+0000 7f4776ffd640 10 monclient: dump: 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.021 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.015+0000 7f477e49c640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.015+0000 7f477e49c640 4 set_mon_vals no callback set 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.015+0000 7f477e49c640 10 monclient: shutdown 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.015+0000 7f477e49c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.015+0000 7f477e49c640 1 -- 10.20.193.189:0/3146813947 >> v1:10.20.193.93:6789/0 conn(0x7f47781406c0 legacy=0x7f4778160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 1 -- 10.20.193.189:0/3146813947 shutdown_connections 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 1 -- 10.20.193.189:0/3146813947 >> 10.20.193.189:0/3146813947 conn(0x7f47780588d0 msgr2=0x7f4778058cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 1 -- 10.20.193.189:0/3146813947 shutdown_connections 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 1 -- 10.20.193.189:0/3146813947 wait complete. 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 10 monclient: build_initial_monmap 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 10 monclient: monmap: 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 1 librados: starting msgr at 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 1 librados: starting objecter 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 1 Processor -- start 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 1 -- start start 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 1 librados: setting wanted keys 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 1 librados: calling monclient init 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 10 monclient: init 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 10 monclient(hunting): picked mon.noname-b con 0x7f477811e790 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 10 monclient(hunting): picked mon.noname-a con 0x7f47781406c0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:00.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 10 monclient(hunting): picked mon.noname-c con 0x7f4778162250 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4778125690 con 0x7f47781406c0 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f477812d460 con 0x7f477811e790 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f477812e660 con 0x7f4778162250 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f477e49c640 10 monclient(hunting): authenticate will time out at 2967.714685s 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f4777fff640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f477811e790 0x7f47781826a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.189:41644/0 (socket says 10.20.193.189:41644) 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.016+0000 7f4777fff640 1 -- 10.20.193.189:0/2241851057 learned_addr learned my addr 10.20.193.189:0/2241851057 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.017+0000 7f4774ff9640 10 client.?.objecter ms_handle_connect 0x7f477811e790 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.017+0000 7f4774ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.017+0000 7f4774ff9640 10 client.?.objecter ms_handle_connect 0x7f4778162250 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.017+0000 7f4774ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.018+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 304519017 0 0) 0x7f477812d460 con 0x7f477811e790 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.018+0000 7f4774ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.018+0000 7f4774ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.018+0000 7f4774ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.018+0000 7f4774ff9640 10 monclient(hunting): my global_id is 4177 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.018+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4748003610 con 0x7f477811e790 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.018+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2421029834 0 0) 0x7f477812e660 con 0x7f4778162250 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.018+0000 7f4774ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.018+0000 7f4774ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.018+0000 7f4774ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.018+0000 7f4774ff9640 10 monclient(hunting): my global_id is 4184 2026-02-25T15:01:00.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.018+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f477812d460 con 0x7f4778162250 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.020+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 892411176 0 0) 0x7f4748003610 con 0x7f477811e790 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.020+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f477812e660 con 0x7f477811e790 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.020+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4764002fc0 con 0x7f477811e790 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.020+0000 7f4774ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.020+0000 7f4774ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.020+0000 7f4774ff9640 10 monclient(hunting): dump: 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.020+0000 7f4774ff9640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.020+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 292593945 0 0) 0x7f477812d460 con 0x7f4778162250 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.020+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4748003610 con 0x7f4778162250 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.020+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f476c003540 con 0x7f4778162250 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.020+0000 7f4774ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.020+0000 7f4774ff9640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.020+0000 7f4774ff9640 10 monclient(hunting): dump: 2026-02-25T15:01:00.024 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.021+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4194387886 0 0) 0x7f477812e660 con 0x7f477811e790 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.021+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 >> v1:10.20.193.94:6789/0 conn(0x7f4778162250 legacy=0x7f4778161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.021+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 >> v1:10.20.193.90:6789/0 conn(0x7f47781406c0 legacy=0x7f4778124c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.021+0000 7f4774ff9640 10 monclient: _finish_hunting 0 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.021+0000 7f4774ff9640 1 monclient: found mon.b 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.021+0000 7f4774ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.021+0000 7f4774ff9640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477812f900 con 0x7f477811e790 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f4774ff9640 10 monclient: _finish_auth 0 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f4774ff9640 10 monclient: _check_auth_tickets 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f4774ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f477e49c640 5 monclient: authenticate success, global_id 4177 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f477e49c640 10 monclient: _renew_subs 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f477e49c640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f477e49c640 1 -- 10.20.193.189:0/2241851057 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f477812e890 con 0x7f477811e790 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f477e49c640 10 client.4177.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f477e49c640 10 monclient: _renew_subs 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f477e49c640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f477e49c640 1 -- 10.20.193.189:0/2241851057 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f477812eed0 con 0x7f477811e790 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f477e49c640 1 librados: init done 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f477e49c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f477e49c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f477e49c640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:00.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.022+0000 7f477e49c640 1 -- 10.20.193.189:0/2241851057 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4778186ab0 con 0x7f477811e790 2026-02-25T15:01:00.055 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.055+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f47640049f0 con 0x7f477811e790 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.055+0000 7f4774ff9640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.055+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4764004d40 con 0x7f477811e790 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.055+0000 7f4774ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.055+0000 7f4774ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.055+0000 7f47767fc640 4 set_mon_vals no callback set 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.055+0000 7f4774ff9640 10 monclient: dump: 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.056 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.055+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 2) ==== 51721+0+0 (unknown 3009573868 0 0) 0x7f4764011900 con 0x7f477811e790 2026-02-25T15:01:00.057 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.056+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(10..10 src has 1..10) ==== 1640+0+0 (unknown 582602725 0 0) 0x7f4764013260 con 0x7f477811e790 2026-02-25T15:01:00.057 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.056+0000 7f4774ff9640 10 client.4177.objecter ms_dispatch2 0x7f4778000b90 osd_map(10..10 src has 1..10) 2026-02-25T15:01:00.057 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.056+0000 7f4774ff9640 3 client.4177.objecter handle_osd_map got epochs [10,10] > 0 2026-02-25T15:01:00.057 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.056+0000 7f4774ff9640 3 client.4177.objecter handle_osd_map decoding full epoch 10 2026-02-25T15:01:00.057 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.056+0000 7f4774ff9640 20 client.4177.objecter dump_active .. 0 homeless 2026-02-25T15:01:00.057 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.057+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77519 (unknown 1092875540 0 440498749) 0x7f4764017970 con 0x7f477811e790 2026-02-25T15:01:00.057 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.057+0000 7f4774ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:00.057 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.057+0000 7f4774ff9640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:00.090 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.089+0000 7f477e49c640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "b292f7c4-9a41-4e48-876c-29281c5ec73b", "id": 8}] 2026-02-25T15:01:00.090 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.089+0000 7f477e49c640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "b292f7c4-9a41-4e48-876c-29281c5ec73b", "id": 8}] 2026-02-25T15:01:00.090 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.089+0000 7f477e49c640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:00.091 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.089+0000 7f477e49c640 1 -- 10.20.193.189:0/2241851057 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd new", "uuid": "b292f7c4-9a41-4e48-876c-29281c5ec73b", "id": 8} v 0) -- 0x7f477812f260 con 0x7f477811e790 2026-02-25T15:01:00.207 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.206+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mgrmap(e 3) ==== 103062+0+0 (unknown 2227658051 0 0) 0x7f4764016b90 con 0x7f477811e790 2026-02-25T15:01:00.208 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.207+0000 7f47777fe640 10 monclient: get_auth_request con 0x7f474807a2c0 auth_method 0 2026-02-25T15:01:00.208 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.207+0000 7f4774ff9640 10 client.4177.objecter ms_handle_connect 0x7f474807a2c0 2026-02-25T15:01:00.238 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-25T15:01:00.237+0000 7fa90d40b640 -1 mgr.server handle_report got status from non-daemon mon.c 2026-02-25T15:01:00.246 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-25T15:01:00.245+0000 7fa90d40b640 -1 mgr.server handle_report got status from non-daemon mon.b 2026-02-25T15:01:00.293 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.291+0000 7f4774ff9640 1 -- 10.20.193.189:0/2241851057 <== mon.1 v1:10.20.193.93:6789/0 11 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b292f7c4-9a41-4e48-876c-29281c5ec73b", "id": 8}]=0 v11) ==== 112+0+2 (unknown 1518148218 0 3065575683) 0x7f4764049a40 con 0x7f477811e790 2026-02-25T15:01:00.293 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.291+0000 7f4774ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "b292f7c4-9a41-4e48-876c-29281c5ec73b", "id": 8}] 2026-02-25T15:01:00.293 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.291+0000 7f4774ff9640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:00.293 INFO:teuthology.orchestra.run.trial189.stdout:8 2026-02-25T15:01:00.295 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.292+0000 7f477e49c640 10 librados: watch_flush enter 2026-02-25T15:01:00.295 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.292+0000 7f477e49c640 10 librados: watch_flush exit 2026-02-25T15:01:00.295 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.292+0000 7f477e49c640 20 client.4177.objecter shutdown clearing up homeless session... 2026-02-25T15:01:00.295 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.292+0000 7f477e49c640 10 client.4177.objecter successfully canceled tick 2026-02-25T15:01:00.295 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.292+0000 7f477e49c640 1 -- 10.20.193.189:0/2241851057 >> v1:10.20.193.189:6800/3647375982 conn(0x7f474807a2c0 legacy=0x7f474809a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.295 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.292+0000 7f477e49c640 10 monclient: shutdown 2026-02-25T15:01:00.295 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.292+0000 7f477e49c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:00.295 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.292+0000 7f477e49c640 1 -- 10.20.193.189:0/2241851057 >> v1:10.20.193.93:6789/0 conn(0x7f477811e790 legacy=0x7f47781826a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.295 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.292+0000 7f477e49c640 1 -- 10.20.193.189:0/2241851057 shutdown_connections 2026-02-25T15:01:00.296 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.292+0000 7f477e49c640 1 -- 10.20.193.189:0/2241851057 >> 10.20.193.189:0/2241851057 conn(0x7f47780588d0 msgr2=0x7f47781831f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:00.296 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.292+0000 7f477e49c640 1 -- 10.20.193.189:0/2241851057 shutdown_connections 2026-02-25T15:01:00.296 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.292+0000 7f477e49c640 1 -- 10.20.193.189:0/2241851057 wait complete. 2026-02-25T15:01:00.296 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.292+0000 7f477e49c640 1 librados: shutdown 2026-02-25T15:01:00.299 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 954a7cbb-2f06-4cb9-ae68-380ac58a9f21 9 2026-02-25T15:01:00.365 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.362+0000 7fb339567640 10 monclient: get_monmap_and_config 2026-02-25T15:01:00.365 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.362+0000 7fb339567640 10 monclient: build_initial_monmap 2026-02-25T15:01:00.365 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.362+0000 7fb339567640 10 monclient: monmap: 2026-02-25T15:01:00.365 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:00.365 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.366 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:01:00.363838+0000 2026-02-25T15:01:00.366 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:01:00.363838+0000 2026-02-25T15:01:00.366 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:00.366 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.366 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:00.366 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:00.366 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:00.366 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.366 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.362+0000 7fb339567640 1 Processor -- start 2026-02-25T15:01:00.366 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.362+0000 7fb339567640 1 -- start start 2026-02-25T15:01:00.367 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.362+0000 7fb339567640 10 monclient: init 2026-02-25T15:01:00.367 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.363+0000 7fb339567640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:00.367 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.363+0000 7fb339567640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:00.367 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.363+0000 7fb339567640 10 monclient(hunting): picked mon.noname-a con 0x7fb33405d9b0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:00.367 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.363+0000 7fb339567640 10 monclient(hunting): picked mon.noname-c con 0x7fb33405dd80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:00.367 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.363+0000 7fb339567640 10 monclient(hunting): picked mon.noname-b con 0x7fb3341649b0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:00.367 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.363+0000 7fb339567640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb33405e200 con 0x7fb33405d9b0 2026-02-25T15:01:00.367 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.363+0000 7fb339567640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3341874c0 con 0x7fb3341649b0 2026-02-25T15:01:00.367 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.363+0000 7fb339567640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3341886a0 con 0x7fb33405dd80 2026-02-25T15:01:00.367 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.363+0000 7fb339567640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:00.368 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.363+0000 7fb339567640 10 monclient(hunting): authenticate will time out at 2968.061220s 2026-02-25T15:01:00.368 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.363+0000 7fb3337fe640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fb3341649b0 0x7fb334184d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.189:41656/0 (socket says 10.20.193.189:41656) 2026-02-25T15:01:00.368 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.363+0000 7fb3337fe640 1 -- 10.20.193.189:0/1658181765 learned_addr learned my addr 10.20.193.189:0/1658181765 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:00.377 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 1 -- 10.20.193.189:0/1658181765 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1495795014 0 0) 0x7fb3341874c0 con 0x7fb3341649b0 2026-02-25T15:01:00.377 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.377 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.377 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.377 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 10 monclient(hunting): my global_id is 4180 2026-02-25T15:01:00.377 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 1 -- 10.20.193.189:0/1658181765 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb320003610 con 0x7fb3341649b0 2026-02-25T15:01:00.377 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 1 -- 10.20.193.189:0/1658181765 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2256186434 0 0) 0x7fb3341886a0 con 0x7fb33405dd80 2026-02-25T15:01:00.377 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.377 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.377 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.378 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 10 monclient(hunting): my global_id is 4187 2026-02-25T15:01:00.378 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 1 -- 10.20.193.189:0/1658181765 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3341874c0 con 0x7fb33405dd80 2026-02-25T15:01:00.378 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 1 -- 10.20.193.189:0/1658181765 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3094709554 0 0) 0x7fb33405e200 con 0x7fb33405d9b0 2026-02-25T15:01:00.378 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.378 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.378 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.378 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 10 monclient(hunting): my global_id is 4176 2026-02-25T15:01:00.378 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.364+0000 7fb331ffb640 1 -- 10.20.193.189:0/1658181765 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3341886a0 con 0x7fb33405d9b0 2026-02-25T15:01:00.378 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb331ffb640 1 -- 10.20.193.189:0/1658181765 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 557865701 0 0) 0x7fb3341886a0 con 0x7fb33405d9b0 2026-02-25T15:01:00.378 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb331ffb640 1 -- 10.20.193.189:0/1658181765 >> v1:10.20.193.94:6789/0 conn(0x7fb33405dd80 legacy=0x7fb33411b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.378 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb331ffb640 1 -- 10.20.193.189:0/1658181765 >> v1:10.20.193.93:6789/0 conn(0x7fb3341649b0 legacy=0x7fb334184d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.378 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb331ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:00.378 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb331ffb640 1 monclient: found mon.noname-a 2026-02-25T15:01:00.378 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb331ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:00.379 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb331ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:00.379 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb331ffb640 1 -- 10.20.193.189:0/1658181765 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb334189920 con 0x7fb33405d9b0 2026-02-25T15:01:00.379 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb331ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:00.379 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb331ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:00.379 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb331ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:00.379 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb331ffb640 1 -- 10.20.193.189:0/1658181765 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb328002dd0 con 0x7fb33405d9b0 2026-02-25T15:01:00.379 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb339567640 5 monclient: authenticate success, global_id 4176 2026-02-25T15:01:00.379 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb339567640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:00.379 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb331ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:00.379 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb331ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:00.379 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb331ffb640 10 monclient: dump: 2026-02-25T15:01:00.379 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.379 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.379 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.379 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.380 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.380 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.380 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.380 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.380 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.380 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.380 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.366+0000 7fb339567640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:00.380 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.367+0000 7fb331ffb640 1 -- 10.20.193.189:0/1658181765 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb328002fe0 con 0x7fb33405d9b0 2026-02-25T15:01:00.380 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.367+0000 7fb331ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:00.380 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.367+0000 7fb331ffb640 1 -- 10.20.193.189:0/1658181765 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb3280033e0 con 0x7fb33405d9b0 2026-02-25T15:01:00.380 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.367+0000 7fb331ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:00.380 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.367+0000 7fb331ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:00.380 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.367+0000 7fb331ffb640 10 monclient: dump: 2026-02-25T15:01:00.380 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.380 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.367+0000 7fb339567640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.367+0000 7fb339567640 4 set_mon_vals no callback set 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.367+0000 7fb339567640 10 monclient: shutdown 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.367+0000 7fb339567640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.367+0000 7fb339567640 1 -- 10.20.193.189:0/1658181765 >> v1:10.20.193.90:6789/0 conn(0x7fb33405d9b0 legacy=0x7fb33411b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.367+0000 7fb339567640 1 -- 10.20.193.189:0/1658181765 shutdown_connections 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.367+0000 7fb339567640 1 -- 10.20.193.189:0/1658181765 >> 10.20.193.189:0/1658181765 conn(0x7fb3340588d0 msgr2=0x7fb334058cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.367+0000 7fb339567640 1 -- 10.20.193.189:0/1658181765 shutdown_connections 2026-02-25T15:01:00.381 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.367+0000 7fb339567640 1 -- 10.20.193.189:0/1658181765 wait complete. 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 10 monclient: build_initial_monmap 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 10 monclient: monmap: 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 1 librados: starting msgr at 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 1 librados: starting objecter 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:00.382 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 1 Processor -- start 2026-02-25T15:01:00.383 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 1 -- start start 2026-02-25T15:01:00.383 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 1 librados: setting wanted keys 2026-02-25T15:01:00.383 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 1 librados: calling monclient init 2026-02-25T15:01:00.383 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 10 monclient: init 2026-02-25T15:01:00.383 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:00.383 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:00.383 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 10 monclient(hunting): picked mon.noname-b con 0x7fb33405d9b0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:00.383 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 10 monclient(hunting): picked mon.noname-a con 0x7fb33405dd80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:00.383 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 10 monclient(hunting): picked mon.noname-c con 0x7fb3341649b0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:00.383 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb334101fb0 con 0x7fb33405dd80 2026-02-25T15:01:00.383 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb334108e90 con 0x7fb33405d9b0 2026-02-25T15:01:00.383 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb33410a070 con 0x7fb3341649b0 2026-02-25T15:01:00.384 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:00.384 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb339567640 10 monclient(hunting): authenticate will time out at 2968.066546s 2026-02-25T15:01:00.384 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb3327fc640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fb33405dd80 0x7fb334161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.189:34430/0 (socket says 10.20.193.189:34430) 2026-02-25T15:01:00.384 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.368+0000 7fb3327fc640 1 -- 10.20.193.189:0/2721123646 learned_addr learned my addr 10.20.193.189:0/2721123646 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:00.384 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.369+0000 7fb3137fe640 10 client.?.objecter ms_handle_connect 0x7fb33405dd80 2026-02-25T15:01:00.384 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.369+0000 7fb3137fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:00.384 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.369+0000 7fb3137fe640 10 client.?.objecter ms_handle_connect 0x7fb3341649b0 2026-02-25T15:01:00.384 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.369+0000 7fb3137fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:00.384 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.369+0000 7fb3137fe640 10 client.?.objecter ms_handle_connect 0x7fb33405d9b0 2026-02-25T15:01:00.384 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.369+0000 7fb3137fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:00.385 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.369+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2394139734 0 0) 0x7fb334101fb0 con 0x7fb33405dd80 2026-02-25T15:01:00.385 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.369+0000 7fb3137fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.385 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.369+0000 7fb3137fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.385 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.369+0000 7fb3137fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.385 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.369+0000 7fb3137fe640 10 monclient(hunting): my global_id is 4179 2026-02-25T15:01:00.385 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.369+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb300003610 con 0x7fb33405dd80 2026-02-25T15:01:00.385 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.370+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3383906787 0 0) 0x7fb33410a070 con 0x7fb3341649b0 2026-02-25T15:01:00.385 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.370+0000 7fb3137fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.385 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.370+0000 7fb3137fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.385 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.370+0000 7fb3137fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.385 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.370+0000 7fb3137fe640 10 monclient(hunting): my global_id is 4190 2026-02-25T15:01:00.385 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.370+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb334101fb0 con 0x7fb3341649b0 2026-02-25T15:01:00.386 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.370+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 90288197 0 0) 0x7fb334108e90 con 0x7fb33405d9b0 2026-02-25T15:01:00.386 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.370+0000 7fb3137fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.386 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.370+0000 7fb3137fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.386 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.370+0000 7fb3137fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.386 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.370+0000 7fb3137fe640 10 monclient(hunting): my global_id is 4183 2026-02-25T15:01:00.386 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.370+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb33410a070 con 0x7fb33405d9b0 2026-02-25T15:01:00.386 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.372+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4106742328 0 0) 0x7fb300003610 con 0x7fb33405dd80 2026-02-25T15:01:00.386 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.372+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb334108e90 con 0x7fb33405dd80 2026-02-25T15:01:00.386 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.372+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb31c0031c0 con 0x7fb33405dd80 2026-02-25T15:01:00.386 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.372+0000 7fb3137fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:00.386 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.372+0000 7fb3137fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:00.386 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.372+0000 7fb3137fe640 10 monclient(hunting): dump: 2026-02-25T15:01:00.387 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.387 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.387 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.387 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.387 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.387 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.387 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.387 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.387 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.387 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.387 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.372+0000 7fb3137fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:00.387 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.372+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2494576338 0 0) 0x7fb334101fb0 con 0x7fb3341649b0 2026-02-25T15:01:00.388 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.372+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb300003610 con 0x7fb3341649b0 2026-02-25T15:01:00.388 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.372+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb324003430 con 0x7fb3341649b0 2026-02-25T15:01:00.388 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.372+0000 7fb3137fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:00.388 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.372+0000 7fb3137fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:00.388 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.372+0000 7fb3137fe640 10 monclient(hunting): dump: 2026-02-25T15:01:00.388 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.388 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.388 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.388 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.388 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.388 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.388 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.389 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.389 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.389 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.389 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3511722843 0 0) 0x7fb33410a070 con 0x7fb33405d9b0 2026-02-25T15:01:00.389 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb334101fb0 con 0x7fb33405d9b0 2026-02-25T15:01:00.389 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb328002ce0 con 0x7fb33405d9b0 2026-02-25T15:01:00.389 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:00.389 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:00.389 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 10 monclient(hunting): dump: 2026-02-25T15:01:00.389 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.389 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.389 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.390 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.390 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.390 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.390 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.390 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.390 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.390 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.390 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3500764597 0 0) 0x7fb334108e90 con 0x7fb33405dd80 2026-02-25T15:01:00.390 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 >> v1:10.20.193.94:6789/0 conn(0x7fb3341649b0 legacy=0x7fb3341630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.390 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 >> v1:10.20.193.93:6789/0 conn(0x7fb33405d9b0 legacy=0x7fb3341014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.390 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:00.390 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 1 monclient: found mon.a 2026-02-25T15:01:00.391 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:00.391 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:00.391 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33410b250 con 0x7fb33405dd80 2026-02-25T15:01:00.391 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:00.391 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:00.391 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb3137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:00.391 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb339567640 5 monclient: authenticate success, global_id 4179 2026-02-25T15:01:00.391 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb339567640 10 monclient: _renew_subs 2026-02-25T15:01:00.391 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb339567640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:00.391 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb339567640 1 -- 10.20.193.189:0/2721123646 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb33410a240 con 0x7fb33405dd80 2026-02-25T15:01:00.391 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb339567640 10 client.4179.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:00.391 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb339567640 10 monclient: _renew_subs 2026-02-25T15:01:00.392 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb339567640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:00.392 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb339567640 1 -- 10.20.193.189:0/2721123646 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb33410a880 con 0x7fb33405dd80 2026-02-25T15:01:00.392 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.373+0000 7fb339567640 1 librados: init done 2026-02-25T15:01:00.392 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.374+0000 7fb339567640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:00.392 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.374+0000 7fb339567640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:00.392 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.374+0000 7fb339567640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:00.392 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.374+0000 7fb339567640 1 -- 10.20.193.189:0/2721123646 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3341869b0 con 0x7fb33405dd80 2026-02-25T15:01:00.465 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.464+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb31c0037b0 con 0x7fb33405dd80 2026-02-25T15:01:00.465 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.464+0000 7fb3137fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:00.465 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.464+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb31c003bc0 con 0x7fb33405dd80 2026-02-25T15:01:00.465 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.464+0000 7fb3137fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:00.465 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.464+0000 7fb3137fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:00.465 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.464+0000 7fb3317fa640 4 set_mon_vals no callback set 2026-02-25T15:01:00.465 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.464+0000 7fb3137fe640 10 monclient: dump: 2026-02-25T15:01:00.466 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.466 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.466 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.466 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.466 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.466 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.466 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.466 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.466 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.466 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.467 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.465+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 3) ==== 103062+0+0 (unknown 2227658051 0 0) 0x7fb31c01e450 con 0x7fb33405dd80 2026-02-25T15:01:00.467 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.465+0000 7fb332ffd640 10 monclient: get_auth_request con 0x7fb300079f40 auth_method 0 2026-02-25T15:01:00.467 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.465+0000 7fb3137fe640 10 client.4179.objecter ms_handle_connect 0x7fb300079f40 2026-02-25T15:01:00.470 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.469+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(11..11 src has 1..11) ==== 1755+0+0 (unknown 2015417732 0 0) 0x7fb31c05ac40 con 0x7fb33405dd80 2026-02-25T15:01:00.470 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.469+0000 7fb3137fe640 10 client.4179.objecter ms_dispatch2 0x7fb334000b90 osd_map(11..11 src has 1..11) 2026-02-25T15:01:00.470 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.469+0000 7fb3137fe640 3 client.4179.objecter handle_osd_map got epochs [11,11] > 0 2026-02-25T15:01:00.470 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.469+0000 7fb3137fe640 3 client.4179.objecter handle_osd_map decoding full epoch 11 2026-02-25T15:01:00.471 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.469+0000 7fb3137fe640 20 client.4179.objecter dump_active .. 0 homeless 2026-02-25T15:01:00.471 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.469+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fb31c05f3b0 con 0x7fb33405dd80 2026-02-25T15:01:00.471 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.469+0000 7fb3137fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:00.471 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.469+0000 7fb3137fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:00.581 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.580+0000 7fb339567640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "954a7cbb-2f06-4cb9-ae68-380ac58a9f21", "id": 9}] 2026-02-25T15:01:00.581 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.580+0000 7fb339567640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "954a7cbb-2f06-4cb9-ae68-380ac58a9f21", "id": 9}] 2026-02-25T15:01:00.581 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.580+0000 7fb339567640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:00.581 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.580+0000 7fb339567640 1 -- 10.20.193.189:0/2721123646 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd new", "uuid": "954a7cbb-2f06-4cb9-ae68-380ac58a9f21", "id": 9} v 0) -- 0x7fb33410abc0 con 0x7fb33405dd80 2026-02-25T15:01:00.600 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb3137fe640 1 -- 10.20.193.189:0/2721123646 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "954a7cbb-2f06-4cb9-ae68-380ac58a9f21", "id": 9}]=0 v12) ==== 112+0+2 (unknown 3994009931 0 2769969524) 0x7fb31c05ecb0 con 0x7fb33405dd80 2026-02-25T15:01:00.600 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb3137fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "954a7cbb-2f06-4cb9-ae68-380ac58a9f21", "id": 9}] 2026-02-25T15:01:00.600 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb3137fe640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:00.600 INFO:teuthology.orchestra.run.trial189.stdout:9 2026-02-25T15:01:00.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb339567640 10 librados: watch_flush enter 2026-02-25T15:01:00.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb339567640 10 librados: watch_flush exit 2026-02-25T15:01:00.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb339567640 20 client.4179.objecter shutdown clearing up homeless session... 2026-02-25T15:01:00.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb339567640 10 client.4179.objecter successfully canceled tick 2026-02-25T15:01:00.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb339567640 1 -- 10.20.193.189:0/2721123646 >> v1:10.20.193.189:6800/3647375982 conn(0x7fb300079f40 legacy=0x7fb30009a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb339567640 10 monclient: shutdown 2026-02-25T15:01:00.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb339567640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:00.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb339567640 1 -- 10.20.193.189:0/2721123646 >> v1:10.20.193.90:6789/0 conn(0x7fb33405dd80 legacy=0x7fb334161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb339567640 1 -- 10.20.193.189:0/2721123646 shutdown_connections 2026-02-25T15:01:00.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb339567640 1 -- 10.20.193.189:0/2721123646 >> 10.20.193.189:0/2721123646 conn(0x7fb3340588d0 msgr2=0x7fb334184d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:00.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb339567640 1 -- 10.20.193.189:0/2721123646 shutdown_connections 2026-02-25T15:01:00.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb339567640 1 -- 10.20.193.189:0/2721123646 wait complete. 2026-02-25T15:01:00.601 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.600+0000 7fb339567640 1 librados: shutdown 2026-02-25T15:01:00.608 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 95ed1c74-8ca5-4317-9ef7-fc657b3cda70 10 2026-02-25T15:01:00.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 10 monclient: get_monmap_and_config 2026-02-25T15:01:00.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 10 monclient: build_initial_monmap 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 10 monclient: monmap: 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:01:00.672498+0000 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:01:00.672498+0000 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 1 Processor -- start 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 1 -- start start 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 10 monclient: init 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 10 monclient(hunting): picked mon.noname-b con 0x7f00a805d9b0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 10 monclient(hunting): picked mon.noname-c con 0x7f00a805dd80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 10 monclient(hunting): picked mon.noname-a con 0x7f00a81649b0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a805e200 con 0x7f00a81649b0 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a81874c0 con 0x7f00a805d9b0 2026-02-25T15:01:00.672 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a81886a0 con 0x7f00a805dd80 2026-02-25T15:01:00.673 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:00.673 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.671+0000 7f00ae583640 10 monclient(hunting): authenticate will time out at 2968.369823s 2026-02-25T15:01:00.673 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.672+0000 7f00a7fff640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f00a805d9b0 0x7f00a811b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.189:41674/0 (socket says 10.20.193.189:41674) 2026-02-25T15:01:00.673 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.672+0000 7f00a7fff640 1 -- 10.20.193.189:0/1470093639 learned_addr learned my addr 10.20.193.189:0/1470093639 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:00.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.721+0000 7f00a6ffd640 1 -- 10.20.193.189:0/1470093639 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 554998771 0 0) 0x7f00a805e200 con 0x7f00a81649b0 2026-02-25T15:01:00.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.721+0000 7f00a6ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.721+0000 7f00a6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.721+0000 7f00a6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.721+0000 7f00a6ffd640 10 monclient(hunting): my global_id is 4182 2026-02-25T15:01:00.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.721+0000 7f00a6ffd640 1 -- 10.20.193.189:0/1470093639 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f008c003610 con 0x7f00a81649b0 2026-02-25T15:01:00.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.722+0000 7f00a6ffd640 1 -- 10.20.193.189:0/1470093639 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1208449701 0 0) 0x7f00a81886a0 con 0x7f00a805dd80 2026-02-25T15:01:00.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.722+0000 7f00a6ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.722+0000 7f00a6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.722+0000 7f00a6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.722+0000 7f00a6ffd640 10 monclient(hunting): my global_id is 4193 2026-02-25T15:01:00.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.722+0000 7f00a6ffd640 1 -- 10.20.193.189:0/1470093639 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00a805e200 con 0x7f00a805dd80 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 1 -- 10.20.193.189:0/1470093639 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1438115765 0 0) 0x7f00a81874c0 con 0x7f00a805d9b0 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 10 monclient(hunting): my global_id is 4186 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 1 -- 10.20.193.189:0/1470093639 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00a81886a0 con 0x7f00a805d9b0 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 1 -- 10.20.193.189:0/1470093639 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2312920437 0 0) 0x7f008c003610 con 0x7f00a81649b0 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 1 -- 10.20.193.189:0/1470093639 >> v1:10.20.193.94:6789/0 conn(0x7f00a805dd80 legacy=0x7f00a811b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 1 -- 10.20.193.189:0/1470093639 >> v1:10.20.193.93:6789/0 conn(0x7f00a805d9b0 legacy=0x7f00a811b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 1 monclient: found mon.noname-a 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 1 -- 10.20.193.189:0/1470093639 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a8189920 con 0x7f00a81649b0 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:00.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00ae583640 5 monclient: authenticate success, global_id 4182 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00ae583640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 1 -- 10.20.193.189:0/1470093639 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f009c003170 con 0x7f00a81649b0 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00a6ffd640 10 monclient: dump: 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.726 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.724+0000 7f00ae583640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.728+0000 7f00a6ffd640 1 -- 10.20.193.189:0/1470093639 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f009c003430 con 0x7f00a81649b0 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.728+0000 7f00a6ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.728+0000 7f00a6ffd640 1 -- 10.20.193.189:0/1470093639 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f009c003830 con 0x7f00a81649b0 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.728+0000 7f00a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.728+0000 7f00a6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.728+0000 7f00a6ffd640 10 monclient: dump: 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.728+0000 7f00ae583640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:00.729 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.728+0000 7f00ae583640 4 set_mon_vals no callback set 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.728+0000 7f00ae583640 10 monclient: shutdown 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.728+0000 7f00ae583640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.728+0000 7f00ae583640 1 -- 10.20.193.189:0/1470093639 >> v1:10.20.193.90:6789/0 conn(0x7f00a81649b0 legacy=0x7f00a8184d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.728+0000 7f00ae583640 1 -- 10.20.193.189:0/1470093639 shutdown_connections 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.728+0000 7f00ae583640 1 -- 10.20.193.189:0/1470093639 >> 10.20.193.189:0/1470093639 conn(0x7f00a80588d0 msgr2=0x7f00a8058cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.728+0000 7f00ae583640 1 -- 10.20.193.189:0/1470093639 shutdown_connections 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.728+0000 7f00ae583640 1 -- 10.20.193.189:0/1470093639 wait complete. 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 10 monclient: build_initial_monmap 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 10 monclient: monmap: 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 1 librados: starting msgr at 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 1 librados: starting objecter 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 1 Processor -- start 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 1 -- start start 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 1 librados: setting wanted keys 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 1 librados: calling monclient init 2026-02-25T15:01:00.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 10 monclient: init 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 10 monclient(hunting): picked mon.noname-a con 0x7f00a805d9b0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 10 monclient(hunting): picked mon.noname-b con 0x7f00a805dd80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 10 monclient(hunting): picked mon.noname-c con 0x7f00a81649b0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a8102360 con 0x7f00a805d9b0 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a8109240 con 0x7f00a805dd80 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a810a420 con 0x7f00a81649b0 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00ae583640 10 monclient(hunting): authenticate will time out at 2968.427690s 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00a7fff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f00a805d9b0 0x7f00a8101850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.189:34454/0 (socket says 10.20.193.189:34454) 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.729+0000 7f00a7fff640 1 -- 10.20.193.189:0/1998436851 learned_addr learned my addr 10.20.193.189:0/1998436851 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.730+0000 7f00a4ff9640 10 client.?.objecter ms_handle_connect 0x7f00a805d9b0 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.730+0000 7f00a4ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.730+0000 7f00a4ff9640 10 client.?.objecter ms_handle_connect 0x7f00a81649b0 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.730+0000 7f00a4ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.730+0000 7f00a4ff9640 10 client.?.objecter ms_handle_connect 0x7f00a805dd80 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.730+0000 7f00a4ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1245598918 0 0) 0x7f00a8102360 con 0x7f00a805d9b0 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.732 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.732 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 10 monclient(hunting): my global_id is 4185 2026-02-25T15:01:00.732 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0078003610 con 0x7f00a805d9b0 2026-02-25T15:01:00.732 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2699920362 0 0) 0x7f00a810a420 con 0x7f00a81649b0 2026-02-25T15:01:00.732 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.732 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.732 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.732 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 10 monclient(hunting): my global_id is 4196 2026-02-25T15:01:00.732 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00a8102360 con 0x7f00a81649b0 2026-02-25T15:01:00.732 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2745969644 0 0) 0x7f00a8109240 con 0x7f00a805dd80 2026-02-25T15:01:00.732 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:00.732 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:00.732 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:00.732 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 10 monclient(hunting): my global_id is 4189 2026-02-25T15:01:00.732 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.731+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00a810a420 con 0x7f00a805dd80 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 793852793 0 0) 0x7f0078003610 con 0x7f00a805d9b0 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f00a8109240 con 0x7f00a805d9b0 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0094003060 con 0x7f00a805d9b0 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 10 monclient(hunting): dump: 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:00.734 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 774806002 0 0) 0x7f00a8102360 con 0x7f00a81649b0 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0078003610 con 0x7f00a81649b0 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f009c003130 con 0x7f00a81649b0 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 10 monclient(hunting): dump: 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3487577123 0 0) 0x7f00a810a420 con 0x7f00a805dd80 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f00a8102360 con 0x7f00a805dd80 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f00980030b0 con 0x7f00a805dd80 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.733+0000 7f00a4ff9640 10 monclient(hunting): dump: 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.735 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2555501732 0 0) 0x7f0078003610 con 0x7f00a81649b0 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 >> v1:10.20.193.93:6789/0 conn(0x7f00a805dd80 legacy=0x7f00a8161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 >> v1:10.20.193.90:6789/0 conn(0x7f00a805d9b0 legacy=0x7f00a8101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00a4ff9640 10 monclient: _finish_hunting 0 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00a4ff9640 1 monclient: found mon.c 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00a4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00a4ff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a810b600 con 0x7f00a81649b0 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00a4ff9640 10 monclient: _finish_auth 0 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00a4ff9640 10 monclient: _check_auth_tickets 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00a4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00ae583640 5 monclient: authenticate success, global_id 4196 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00ae583640 10 monclient: _renew_subs 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00ae583640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00ae583640 1 -- 10.20.193.189:0/1998436851 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f00a810a5f0 con 0x7f00a81649b0 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00ae583640 10 client.4196.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00ae583640 10 monclient: _renew_subs 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00ae583640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00ae583640 1 -- 10.20.193.189:0/1998436851 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f00a810ab10 con 0x7f00a81649b0 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00ae583640 1 librados: init done 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00ae583640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00ae583640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00ae583640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:00.736 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.735+0000 7f00ae583640 1 -- 10.20.193.189:0/1998436851 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a8188d70 con 0x7f00a81649b0 2026-02-25T15:01:00.740 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.740+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f009c002c70 con 0x7f00a81649b0 2026-02-25T15:01:00.740 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.740+0000 7f00a4ff9640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:00.740 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.740+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f009c003dd0 con 0x7f00a81649b0 2026-02-25T15:01:00.740 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.740+0000 7f00a4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:00.740 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.740+0000 7f00a4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:00.740 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.740+0000 7f00a4ff9640 10 monclient: dump: 2026-02-25T15:01:00.740 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:00.740 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:00.740 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.740 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:00.740 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:00.741 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:00.741 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:00.741 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:00.741 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:00.741 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:00.741 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.740+0000 7f00a67fc640 4 set_mon_vals no callback set 2026-02-25T15:01:00.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.747+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 3) ==== 103062+0+0 (unknown 2227658051 0 0) 0x7f009c01d070 con 0x7f00a81649b0 2026-02-25T15:01:00.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.748+0000 7f00a7fff640 10 monclient: get_auth_request con 0x7f0078079f10 auth_method 0 2026-02-25T15:01:00.749 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.748+0000 7f00a4ff9640 10 client.4196.objecter ms_handle_connect 0x7f0078079f10 2026-02-25T15:01:00.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.750+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(12..12 src has 1..12) ==== 1870+0+0 (unknown 501486041 0 0) 0x7f009c059ac0 con 0x7f00a81649b0 2026-02-25T15:01:00.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.750+0000 7f00a4ff9640 10 client.4196.objecter ms_dispatch2 0x7f00a8000b90 osd_map(12..12 src has 1..12) 2026-02-25T15:01:00.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.750+0000 7f00a4ff9640 3 client.4196.objecter handle_osd_map got epochs [12,12] > 0 2026-02-25T15:01:00.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.750+0000 7f00a4ff9640 3 client.4196.objecter handle_osd_map decoding full epoch 12 2026-02-25T15:01:00.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.750+0000 7f00a4ff9640 20 client.4196.objecter dump_active .. 0 homeless 2026-02-25T15:01:00.835 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.834+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f009c05e230 con 0x7f00a81649b0 2026-02-25T15:01:00.835 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.834+0000 7f00a4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:00.835 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.834+0000 7f00a4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:00.946 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.945+0000 7f00ae583640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "95ed1c74-8ca5-4317-9ef7-fc657b3cda70", "id": 10}] 2026-02-25T15:01:00.946 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.945+0000 7f00ae583640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "95ed1c74-8ca5-4317-9ef7-fc657b3cda70", "id": 10}] 2026-02-25T15:01:00.946 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.945+0000 7f00ae583640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:00.947 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:00.945+0000 7f00ae583640 1 -- 10.20.193.189:0/1998436851 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd new", "uuid": "95ed1c74-8ca5-4317-9ef7-fc657b3cda70", "id": 10} v 0) -- 0x7f00a810b1e0 con 0x7f00a81649b0 2026-02-25T15:01:01.021 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.021+0000 7f00a4ff9640 1 -- 10.20.193.189:0/1998436851 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "95ed1c74-8ca5-4317-9ef7-fc657b3cda70", "id": 10}]=0 v13) ==== 113+0+3 (unknown 4224505910 0 520826459) 0x7f009c05db30 con 0x7f00a81649b0 2026-02-25T15:01:01.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.021+0000 7f00a4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "95ed1c74-8ca5-4317-9ef7-fc657b3cda70", "id": 10}] 2026-02-25T15:01:01.022 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.021+0000 7f00a4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:01.022 INFO:teuthology.orchestra.run.trial189.stdout:10 2026-02-25T15:01:01.023 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.022+0000 7f00ae583640 10 librados: watch_flush enter 2026-02-25T15:01:01.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.022+0000 7f00ae583640 10 librados: watch_flush exit 2026-02-25T15:01:01.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.022+0000 7f00ae583640 20 client.4196.objecter shutdown clearing up homeless session... 2026-02-25T15:01:01.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.022+0000 7f00ae583640 10 client.4196.objecter successfully canceled tick 2026-02-25T15:01:01.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.022+0000 7f00ae583640 1 -- 10.20.193.189:0/1998436851 >> v1:10.20.193.189:6800/3647375982 conn(0x7f0078079f10 legacy=0x7f007809a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.022+0000 7f00ae583640 10 monclient: shutdown 2026-02-25T15:01:01.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.022+0000 7f00ae583640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:01.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.022+0000 7f00ae583640 1 -- 10.20.193.189:0/1998436851 >> v1:10.20.193.94:6789/0 conn(0x7f00a81649b0 legacy=0x7f00a81630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.022+0000 7f00ae583640 1 -- 10.20.193.189:0/1998436851 shutdown_connections 2026-02-25T15:01:01.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.022+0000 7f00ae583640 1 -- 10.20.193.189:0/1998436851 >> 10.20.193.189:0/1998436851 conn(0x7f00a80588d0 msgr2=0x7f00a80ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:01.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.022+0000 7f00ae583640 1 -- 10.20.193.189:0/1998436851 shutdown_connections 2026-02-25T15:01:01.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.022+0000 7f00ae583640 1 -- 10.20.193.189:0/1998436851 wait complete. 2026-02-25T15:01:01.025 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.022+0000 7f00ae583640 1 librados: shutdown 2026-02-25T15:01:01.033 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 8f617b59-e8d1-42fb-8787-64e8c2117954 11 2026-02-25T15:01:01.097 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.096+0000 7f149bbbb640 10 monclient: get_monmap_and_config 2026-02-25T15:01:01.097 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.096+0000 7f149bbbb640 10 monclient: build_initial_monmap 2026-02-25T15:01:01.097 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.096+0000 7f149bbbb640 10 monclient: monmap: 2026-02-25T15:01:01.097 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:01.097 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.097 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:01:01.097535+0000 2026-02-25T15:01:01.097 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:01:01.097535+0000 2026-02-25T15:01:01.098 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:01.098 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.098 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:01.098 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:01.098 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:01.098 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.100 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.096+0000 7f149bbbb640 1 Processor -- start 2026-02-25T15:01:01.101 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.096+0000 7f149bbbb640 1 -- start start 2026-02-25T15:01:01.101 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.096+0000 7f149bbbb640 10 monclient: init 2026-02-25T15:01:01.101 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.096+0000 7f149bbbb640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:01.101 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.096+0000 7f149bbbb640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:01.101 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.096+0000 7f149bbbb640 10 monclient(hunting): picked mon.noname-b con 0x7f149411d730 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:01.101 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.096+0000 7f149bbbb640 10 monclient(hunting): picked mon.noname-a con 0x7f14941406b0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:01.101 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.096+0000 7f149bbbb640 10 monclient(hunting): picked mon.noname-c con 0x7f1494162240 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:01.101 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.096+0000 7f149bbbb640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1494183fd0 con 0x7f14941406b0 2026-02-25T15:01:01.101 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.097+0000 7f149bbbb640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14941851d0 con 0x7f149411d730 2026-02-25T15:01:01.102 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.097+0000 7f149bbbb640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14941863d0 con 0x7f1494162240 2026-02-25T15:01:01.102 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.097+0000 7f149bbbb640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:01.102 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.097+0000 7f149bbbb640 10 monclient(hunting): authenticate will time out at 2968.795050s 2026-02-25T15:01:01.102 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.097+0000 7f1499930640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f149411d730 0x7f149411db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.189:41692/0 (socket says 10.20.193.189:41692) 2026-02-25T15:01:01.102 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.097+0000 7f1499930640 1 -- 10.20.193.189:0/1995030365 learned_addr learned my addr 10.20.193.189:0/1995030365 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:01.102 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 1 -- 10.20.193.189:0/1995030365 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2047424458 0 0) 0x7f14941851d0 con 0x7f149411d730 2026-02-25T15:01:01.102 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 10 monclient(hunting): _negotiate 2026-02-25T15:01:01.102 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:01.102 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:01.102 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 10 monclient(hunting): my global_id is 4192 2026-02-25T15:01:01.103 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 1 -- 10.20.193.189:0/1995030365 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f147c003610 con 0x7f149411d730 2026-02-25T15:01:01.103 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 1 -- 10.20.193.189:0/1995030365 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 719036799 0 0) 0x7f1494183fd0 con 0x7f14941406b0 2026-02-25T15:01:01.103 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 10 monclient(hunting): _negotiate 2026-02-25T15:01:01.103 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:01.103 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:01.103 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 10 monclient(hunting): my global_id is 4188 2026-02-25T15:01:01.103 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 1 -- 10.20.193.189:0/1995030365 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14941851d0 con 0x7f14941406b0 2026-02-25T15:01:01.103 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 1 -- 10.20.193.189:0/1995030365 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4148221631 0 0) 0x7f14941863d0 con 0x7f1494162240 2026-02-25T15:01:01.103 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 10 monclient(hunting): _negotiate 2026-02-25T15:01:01.103 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:01.103 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:01.103 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 10 monclient(hunting): my global_id is 4199 2026-02-25T15:01:01.103 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.098+0000 7f149892e640 1 -- 10.20.193.189:0/1995030365 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1494183fd0 con 0x7f1494162240 2026-02-25T15:01:01.111 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149892e640 1 -- 10.20.193.189:0/1995030365 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 949871952 0 0) 0x7f1494183fd0 con 0x7f1494162240 2026-02-25T15:01:01.112 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149892e640 1 -- 10.20.193.189:0/1995030365 >> v1:10.20.193.93:6789/0 conn(0x7f149411d730 legacy=0x7f149411db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.112 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149892e640 1 -- 10.20.193.189:0/1995030365 >> v1:10.20.193.90:6789/0 conn(0x7f14941406b0 legacy=0x7f1494160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.112 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149892e640 10 monclient: _finish_hunting 0 2026-02-25T15:01:01.112 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149892e640 1 monclient: found mon.noname-c 2026-02-25T15:01:01.112 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149892e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:01.112 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149892e640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:01.112 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149892e640 1 -- 10.20.193.189:0/1995030365 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1494187670 con 0x7f1494162240 2026-02-25T15:01:01.112 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149892e640 10 monclient: _finish_auth 0 2026-02-25T15:01:01.112 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149892e640 10 monclient: _check_auth_tickets 2026-02-25T15:01:01.113 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149892e640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:01.113 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149bbbb640 5 monclient: authenticate success, global_id 4199 2026-02-25T15:01:01.113 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149bbbb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:01.113 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149892e640 1 -- 10.20.193.189:0/1995030365 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f14900030d0 con 0x7f1494162240 2026-02-25T15:01:01.113 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149892e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:01.113 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149892e640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:01.113 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149892e640 10 monclient: dump: 2026-02-25T15:01:01.113 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.113 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.113 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.113 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.113 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.113 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.114 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.114 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.114 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.114 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.114 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.100+0000 7f149bbbb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:01.114 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.101+0000 7f149892e640 1 -- 10.20.193.189:0/1995030365 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1490003390 con 0x7f1494162240 2026-02-25T15:01:01.114 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.101+0000 7f149892e640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:01.114 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.101+0000 7f149892e640 1 -- 10.20.193.189:0/1995030365 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1490003790 con 0x7f1494162240 2026-02-25T15:01:01.114 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.101+0000 7f149bbbb640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:01.114 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.101+0000 7f149bbbb640 4 set_mon_vals no callback set 2026-02-25T15:01:01.114 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.101+0000 7f149892e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:01.115 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.101+0000 7f149892e640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:01.115 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.101+0000 7f149892e640 10 monclient: dump: 2026-02-25T15:01:01.115 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.115 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.115 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.115 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.115 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.115 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.115 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.115 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.115 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.115 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.116 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.101+0000 7f149bbbb640 10 monclient: shutdown 2026-02-25T15:01:01.116 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.101+0000 7f149bbbb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:01.116 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.101+0000 7f149bbbb640 1 -- 10.20.193.189:0/1995030365 >> v1:10.20.193.94:6789/0 conn(0x7f1494162240 legacy=0x7f14941826d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.116 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.101+0000 7f149bbbb640 1 -- 10.20.193.189:0/1995030365 shutdown_connections 2026-02-25T15:01:01.116 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.101+0000 7f149bbbb640 1 -- 10.20.193.189:0/1995030365 >> 10.20.193.189:0/1995030365 conn(0x7f1494056eb0 msgr2=0x7f14940572b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:01.116 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.101+0000 7f149bbbb640 1 -- 10.20.193.189:0/1995030365 shutdown_connections 2026-02-25T15:01:01.116 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.101+0000 7f149bbbb640 1 -- 10.20.193.189:0/1995030365 wait complete. 2026-02-25T15:01:01.116 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 10 monclient: build_initial_monmap 2026-02-25T15:01:01.116 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 10 monclient: monmap: 2026-02-25T15:01:01.116 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:01.116 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:01.116 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:01:01.117 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:01:01.117 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:01.117 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:01:01.117 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:01.117 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:01.117 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:01.117 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.117 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 1 librados: starting msgr at 2026-02-25T15:01:01.117 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 1 librados: starting objecter 2026-02-25T15:01:01.117 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:01.117 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 1 Processor -- start 2026-02-25T15:01:01.117 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 1 -- start start 2026-02-25T15:01:01.117 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 1 librados: setting wanted keys 2026-02-25T15:01:01.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 1 librados: calling monclient init 2026-02-25T15:01:01.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 10 monclient: init 2026-02-25T15:01:01.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:01.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:01.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 10 monclient(hunting): picked mon.noname-c con 0x7f149411d730 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:01.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 10 monclient(hunting): picked mon.noname-a con 0x7f14941406b0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:01.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 10 monclient(hunting): picked mon.noname-b con 0x7f1494162240 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:01.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f149412e580 con 0x7f14941406b0 2026-02-25T15:01:01.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f149412f780 con 0x7f1494162240 2026-02-25T15:01:01.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1494130980 con 0x7f149411d730 2026-02-25T15:01:01.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:01.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149bbbb640 10 monclient(hunting): authenticate will time out at 2968.800400s 2026-02-25T15:01:01.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149a131640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f1494162240 0x7f149412cd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.189:41708/0 (socket says 10.20.193.189:41708) 2026-02-25T15:01:01.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.102+0000 7f149a131640 1 -- 10.20.193.189:0/479939188 learned_addr learned my addr 10.20.193.189:0/479939188 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:01.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.103+0000 7f14827fc640 10 client.?.objecter ms_handle_connect 0x7f1494162240 2026-02-25T15:01:01.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.103+0000 7f14827fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:01.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.103+0000 7f14827fc640 10 client.?.objecter ms_handle_connect 0x7f149411d730 2026-02-25T15:01:01.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.103+0000 7f14827fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:01.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.103+0000 7f14827fc640 10 client.?.objecter ms_handle_connect 0x7f14941406b0 2026-02-25T15:01:01.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.103+0000 7f14827fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:01.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2853421584 0 0) 0x7f149412f780 con 0x7f1494162240 2026-02-25T15:01:01.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:01.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:01.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:01.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 10 monclient(hunting): my global_id is 4195 2026-02-25T15:01:01.120 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1464003610 con 0x7f1494162240 2026-02-25T15:01:01.120 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1251665902 0 0) 0x7f1494130980 con 0x7f149411d730 2026-02-25T15:01:01.120 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:01.120 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:01.120 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:01.120 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 10 monclient(hunting): my global_id is 4202 2026-02-25T15:01:01.120 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f149412f780 con 0x7f149411d730 2026-02-25T15:01:01.120 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3915970967 0 0) 0x7f149412e580 con 0x7f14941406b0 2026-02-25T15:01:01.120 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:01.120 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:01.120 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:01.120 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 10 monclient(hunting): my global_id is 4191 2026-02-25T15:01:01.121 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.104+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1494130980 con 0x7f14941406b0 2026-02-25T15:01:01.121 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.106+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1597811879 0 0) 0x7f1464003610 con 0x7f1494162240 2026-02-25T15:01:01.121 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.106+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f149412e580 con 0x7f1494162240 2026-02-25T15:01:01.121 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.106+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1490003090 con 0x7f1494162240 2026-02-25T15:01:01.121 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.106+0000 7f14827fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:01.121 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.106+0000 7f14827fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:01.121 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.106+0000 7f14827fc640 10 monclient(hunting): dump: 2026-02-25T15:01:01.121 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.121 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.121 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.121 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.121 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.121 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.122 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.122 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.122 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.122 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.122 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.106+0000 7f14827fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:01.122 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.106+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4062598048 0 0) 0x7f1494130980 con 0x7f14941406b0 2026-02-25T15:01:01.122 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.106+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1464003610 con 0x7f14941406b0 2026-02-25T15:01:01.122 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.106+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f14880030b0 con 0x7f14941406b0 2026-02-25T15:01:01.122 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.106+0000 7f14827fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:01.122 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.106+0000 7f14827fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:01.122 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.106+0000 7f14827fc640 10 monclient(hunting): dump: 2026-02-25T15:01:01.122 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.123 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.123 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.123 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.123 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.123 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.123 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.123 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.123 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.123 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.123 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.106+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3351000021 0 0) 0x7f149412f780 con 0x7f149411d730 2026-02-25T15:01:01.123 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.107+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1494130980 con 0x7f149411d730 2026-02-25T15:01:01.123 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.107+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f14840030b0 con 0x7f149411d730 2026-02-25T15:01:01.124 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.107+0000 7f14827fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:01.124 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.107+0000 7f14827fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:01.124 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.107+0000 7f14827fc640 10 monclient(hunting): dump: 2026-02-25T15:01:01.124 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.124 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.124 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.124 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.124 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.124 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.124 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.124 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.124 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.124 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.125 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1804400555 0 0) 0x7f149412e580 con 0x7f1494162240 2026-02-25T15:01:01.125 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 >> v1:10.20.193.94:6789/0 conn(0x7f149411d730 legacy=0x7f14941826b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.125 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 >> v1:10.20.193.90:6789/0 conn(0x7f14941406b0 legacy=0x7f1494160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.125 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f14827fc640 10 monclient: _finish_hunting 0 2026-02-25T15:01:01.125 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f14827fc640 1 monclient: found mon.b 2026-02-25T15:01:01.125 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f14827fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:01.125 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f14827fc640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:01.125 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1494131c20 con 0x7f1494162240 2026-02-25T15:01:01.125 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f14827fc640 10 monclient: _finish_auth 0 2026-02-25T15:01:01.125 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f14827fc640 10 monclient: _check_auth_tickets 2026-02-25T15:01:01.125 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f14827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:01.125 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f149bbbb640 5 monclient: authenticate success, global_id 4195 2026-02-25T15:01:01.125 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f149bbbb640 10 monclient: _renew_subs 2026-02-25T15:01:01.126 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f149bbbb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:01.126 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f149bbbb640 1 -- 10.20.193.189:0/479939188 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1494130bb0 con 0x7f1494162240 2026-02-25T15:01:01.126 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f149bbbb640 10 client.4195.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:01.126 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f149bbbb640 10 monclient: _renew_subs 2026-02-25T15:01:01.126 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f149bbbb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:01.126 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f149bbbb640 1 -- 10.20.193.189:0/479939188 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f14941311a0 con 0x7f1494162240 2026-02-25T15:01:01.126 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f149bbbb640 1 librados: init done 2026-02-25T15:01:01.126 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f149bbbb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:01.126 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f149bbbb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:01.126 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f149bbbb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:01.126 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.108+0000 7f149bbbb640 1 -- 10.20.193.189:0/479939188 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1494184530 con 0x7f1494162240 2026-02-25T15:01:01.128 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.113+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f14900032a0 con 0x7f1494162240 2026-02-25T15:01:01.128 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.113+0000 7f14827fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:01.128 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.113+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1490003e90 con 0x7f1494162240 2026-02-25T15:01:01.128 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.113+0000 7f14827fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:01.128 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.113+0000 7f14827fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:01.128 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.113+0000 7f14827fc640 10 monclient: dump: 2026-02-25T15:01:01.128 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.128 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.128 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.128 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.129 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.129 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.129 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.129 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.129 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.129 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.129 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.113+0000 7f1483fff640 4 set_mon_vals no callback set 2026-02-25T15:01:01.129 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.116+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 3) ==== 103062+0+0 (unknown 2227658051 0 0) 0x7f1490003e90 con 0x7f1494162240 2026-02-25T15:01:01.129 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.117+0000 7f1499930640 10 monclient: get_auth_request con 0x7f1464079f40 auth_method 0 2026-02-25T15:01:01.129 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.117+0000 7f14827fc640 10 client.4195.objecter ms_handle_connect 0x7f1464079f40 2026-02-25T15:01:01.129 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.117+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(13..13 src has 1..13) ==== 1985+0+0 (unknown 3122409699 0 0) 0x7f149005af00 con 0x7f1494162240 2026-02-25T15:01:01.129 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.117+0000 7f14827fc640 10 client.4195.objecter ms_dispatch2 0x7f1494000b90 osd_map(13..13 src has 1..13) 2026-02-25T15:01:01.130 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.117+0000 7f14827fc640 3 client.4195.objecter handle_osd_map got epochs [13,13] > 0 2026-02-25T15:01:01.130 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.117+0000 7f14827fc640 3 client.4195.objecter handle_osd_map decoding full epoch 13 2026-02-25T15:01:01.130 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.118+0000 7f14827fc640 20 client.4195.objecter dump_active .. 0 homeless 2026-02-25T15:01:01.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.203+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f149005f670 con 0x7f1494162240 2026-02-25T15:01:01.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.203+0000 7f14827fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:01.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.203+0000 7f14827fc640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:01.314 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.313+0000 7f149bbbb640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "8f617b59-e8d1-42fb-8787-64e8c2117954", "id": 11}] 2026-02-25T15:01:01.314 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.313+0000 7f149bbbb640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "8f617b59-e8d1-42fb-8787-64e8c2117954", "id": 11}] 2026-02-25T15:01:01.314 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.313+0000 7f149bbbb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:01.314 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.313+0000 7f149bbbb640 1 -- 10.20.193.189:0/479939188 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd new", "uuid": "8f617b59-e8d1-42fb-8787-64e8c2117954", "id": 11} v 0) -- 0x7f1494184700 con 0x7f1494162240 2026-02-25T15:01:01.356 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.355+0000 7f14827fc640 1 -- 10.20.193.189:0/479939188 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8f617b59-e8d1-42fb-8787-64e8c2117954", "id": 11}]=0 v14) ==== 113+0+3 (unknown 3914031428 0 212448812) 0x7f149005ef70 con 0x7f1494162240 2026-02-25T15:01:01.356 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.355+0000 7f14827fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "8f617b59-e8d1-42fb-8787-64e8c2117954", "id": 11}] 2026-02-25T15:01:01.356 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.355+0000 7f14827fc640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:01.356 INFO:teuthology.orchestra.run.trial189.stdout:11 2026-02-25T15:01:01.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.356+0000 7f149bbbb640 10 librados: watch_flush enter 2026-02-25T15:01:01.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.356+0000 7f149bbbb640 10 librados: watch_flush exit 2026-02-25T15:01:01.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.356+0000 7f149bbbb640 20 client.4195.objecter shutdown clearing up homeless session... 2026-02-25T15:01:01.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.356+0000 7f149bbbb640 10 client.4195.objecter successfully canceled tick 2026-02-25T15:01:01.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.356+0000 7f149bbbb640 1 -- 10.20.193.189:0/479939188 >> v1:10.20.193.189:6800/3647375982 conn(0x7f1464079f40 legacy=0x7f146409a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.356+0000 7f149bbbb640 10 monclient: shutdown 2026-02-25T15:01:01.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.356+0000 7f149bbbb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:01.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.356+0000 7f149bbbb640 1 -- 10.20.193.189:0/479939188 >> v1:10.20.193.93:6789/0 conn(0x7f1494162240 legacy=0x7f149412cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.356+0000 7f149bbbb640 1 -- 10.20.193.189:0/479939188 shutdown_connections 2026-02-25T15:01:01.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.356+0000 7f149bbbb640 1 -- 10.20.193.189:0/479939188 >> 10.20.193.189:0/479939188 conn(0x7f1494056eb0 msgr2=0x7f149411b700 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:01.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.356+0000 7f149bbbb640 1 -- 10.20.193.189:0/479939188 shutdown_connections 2026-02-25T15:01:01.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.356+0000 7f149bbbb640 1 -- 10.20.193.189:0/479939188 wait complete. 2026-02-25T15:01:01.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.356+0000 7f149bbbb640 1 librados: shutdown 2026-02-25T15:01:01.366 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 4bad4371-2ca7-4023-8406-0275011eda9c 12 2026-02-25T15:01:01.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 10 monclient: get_monmap_and_config 2026-02-25T15:01:01.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 10 monclient: build_initial_monmap 2026-02-25T15:01:01.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 10 monclient: monmap: 2026-02-25T15:01:01.430 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:01.430 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.430 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:01:01.429561+0000 2026-02-25T15:01:01.430 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:01:01.429561+0000 2026-02-25T15:01:01.431 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:01.431 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.431 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:01.431 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:01.431 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:01.431 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 1 Processor -- start 2026-02-25T15:01:01.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 1 -- start start 2026-02-25T15:01:01.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 10 monclient: init 2026-02-25T15:01:01.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:01.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:01.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 10 monclient(hunting): picked mon.noname-a con 0x7f11bc11e790 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:01.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 10 monclient(hunting): picked mon.noname-c con 0x7f11bc1406c0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:01.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 10 monclient(hunting): picked mon.noname-b con 0x7f11bc162250 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:01.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11bc183fe0 con 0x7f11bc11e790 2026-02-25T15:01:01.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11bc1851e0 con 0x7f11bc162250 2026-02-25T15:01:01.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11bc1863e0 con 0x7f11bc1406c0 2026-02-25T15:01:01.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:01.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.428+0000 7f11c0c7b640 10 monclient(hunting): authenticate will time out at 2969.126871s 2026-02-25T15:01:01.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.429+0000 7f11bad76640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f11bc162250 0x7f11bc1826e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.189:41716/0 (socket says 10.20.193.189:41716) 2026-02-25T15:01:01.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.429+0000 7f11bad76640 1 -- 10.20.193.189:0/3538745498 learned_addr learned my addr 10.20.193.189:0/3538745498 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:01.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 1 -- 10.20.193.189:0/3538745498 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3762029309 0 0) 0x7f11bc1863e0 con 0x7f11bc1406c0 2026-02-25T15:01:01.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:01.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:01.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:01.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 10 monclient(hunting): my global_id is 4205 2026-02-25T15:01:01.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 1 -- 10.20.193.189:0/3538745498 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11a8003610 con 0x7f11bc1406c0 2026-02-25T15:01:01.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 1 -- 10.20.193.189:0/3538745498 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1584877633 0 0) 0x7f11bc1851e0 con 0x7f11bc162250 2026-02-25T15:01:01.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:01.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:01.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:01.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 10 monclient(hunting): my global_id is 4198 2026-02-25T15:01:01.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 1 -- 10.20.193.189:0/3538745498 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11bc1863e0 con 0x7f11bc162250 2026-02-25T15:01:01.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 1 -- 10.20.193.189:0/3538745498 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3957722842 0 0) 0x7f11bc183fe0 con 0x7f11bc11e790 2026-02-25T15:01:01.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:01.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:01.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:01.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 10 monclient(hunting): my global_id is 4194 2026-02-25T15:01:01.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.430+0000 7f11b9573640 1 -- 10.20.193.189:0/3538745498 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11bc1851e0 con 0x7f11bc11e790 2026-02-25T15:01:01.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11b9573640 1 -- 10.20.193.189:0/3538745498 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 647569738 0 0) 0x7f11a8003610 con 0x7f11bc1406c0 2026-02-25T15:01:01.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11b9573640 1 -- 10.20.193.189:0/3538745498 >> v1:10.20.193.93:6789/0 conn(0x7f11bc162250 legacy=0x7f11bc1826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11b9573640 1 -- 10.20.193.189:0/3538745498 >> v1:10.20.193.90:6789/0 conn(0x7f11bc11e790 legacy=0x7f11bc11eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11b9573640 10 monclient: _finish_hunting 0 2026-02-25T15:01:01.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11b9573640 1 monclient: found mon.noname-c 2026-02-25T15:01:01.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11b9573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:01.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11b9573640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:01.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11b9573640 1 -- 10.20.193.189:0/3538745498 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc187680 con 0x7f11bc1406c0 2026-02-25T15:01:01.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11b9573640 10 monclient: _finish_auth 0 2026-02-25T15:01:01.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11b9573640 10 monclient: _check_auth_tickets 2026-02-25T15:01:01.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11b9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:01.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11b9573640 1 -- 10.20.193.189:0/3538745498 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f11a4002df0 con 0x7f11bc1406c0 2026-02-25T15:01:01.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11b9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:01.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11b9573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:01.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11b9573640 10 monclient: dump: 2026-02-25T15:01:01.441 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.441 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.441 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.441 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.441 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.441 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.441 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.441 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.441 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.441 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11c0c7b640 5 monclient: authenticate success, global_id 4205 2026-02-25T15:01:01.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.432+0000 7f11c0c7b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:01.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11b9573640 1 -- 10.20.193.189:0/3538745498 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f11a4003030 con 0x7f11bc1406c0 2026-02-25T15:01:01.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11b9573640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:01.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11b9573640 1 -- 10.20.193.189:0/3538745498 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f11a4003470 con 0x7f11bc1406c0 2026-02-25T15:01:01.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11b9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:01.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11b9573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:01.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11b9573640 10 monclient: dump: 2026-02-25T15:01:01.442 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.442 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.442 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.442 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.442 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.442 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.442 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.443 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.443 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.443 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11c0c7b640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:01.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11c0c7b640 4 set_mon_vals no callback set 2026-02-25T15:01:01.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11c0c7b640 10 monclient: shutdown 2026-02-25T15:01:01.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11c0c7b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:01.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11c0c7b640 1 -- 10.20.193.189:0/3538745498 >> v1:10.20.193.94:6789/0 conn(0x7f11bc1406c0 legacy=0x7f11bc160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11c0c7b640 1 -- 10.20.193.189:0/3538745498 shutdown_connections 2026-02-25T15:01:01.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11c0c7b640 1 -- 10.20.193.189:0/3538745498 >> 10.20.193.189:0/3538745498 conn(0x7f11bc0588d0 msgr2=0x7f11bc058cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:01.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11c0c7b640 1 -- 10.20.193.189:0/3538745498 shutdown_connections 2026-02-25T15:01:01.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11c0c7b640 1 -- 10.20.193.189:0/3538745498 wait complete. 2026-02-25T15:01:01.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11c0c7b640 10 monclient: build_initial_monmap 2026-02-25T15:01:01.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11c0c7b640 10 monclient: monmap: 2026-02-25T15:01:01.444 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:01.444 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:01.444 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:01:01.444 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:01:01.444 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:01.444 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:01:01.444 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:01.444 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:01.444 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:01.444 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11c0c7b640 1 librados: starting msgr at 2026-02-25T15:01:01.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11c0c7b640 1 librados: starting objecter 2026-02-25T15:01:01.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11c0c7b640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:01.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.433+0000 7f11c0c7b640 1 Processor -- start 2026-02-25T15:01:01.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11c0c7b640 1 -- start start 2026-02-25T15:01:01.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11c0c7b640 1 librados: setting wanted keys 2026-02-25T15:01:01.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11c0c7b640 1 librados: calling monclient init 2026-02-25T15:01:01.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11c0c7b640 10 monclient: init 2026-02-25T15:01:01.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11c0c7b640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:01.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11c0c7b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:01.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11c0c7b640 10 monclient(hunting): picked mon.noname-a con 0x7f11bc11e790 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:01.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11c0c7b640 10 monclient(hunting): picked mon.noname-c con 0x7f11bc1406c0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:01.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11c0c7b640 10 monclient(hunting): picked mon.noname-b con 0x7f11bc162250 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:01.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11c0c7b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11bc1258f0 con 0x7f11bc11e790 2026-02-25T15:01:01.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11c0c7b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11bc12d750 con 0x7f11bc162250 2026-02-25T15:01:01.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11c0c7b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11bc12e970 con 0x7f11bc1406c0 2026-02-25T15:01:01.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11c0c7b640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:01.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11c0c7b640 10 monclient(hunting): authenticate will time out at 2969.132182s 2026-02-25T15:01:01.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11b9d74640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f11bc1406c0 0x7f11bc124f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.189:32868/0 (socket says 10.20.193.189:32868) 2026-02-25T15:01:01.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11b9d74640 1 -- 10.20.193.189:0/1966445081 learned_addr learned my addr 10.20.193.189:0/1966445081 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:01.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11a2ffd640 10 client.?.objecter ms_handle_connect 0x7f11bc1406c0 2026-02-25T15:01:01.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.434+0000 7f11a2ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:01.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 10 client.?.objecter ms_handle_connect 0x7f11bc11e790 2026-02-25T15:01:01.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:01.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 10 client.?.objecter ms_handle_connect 0x7f11bc162250 2026-02-25T15:01:01.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:01.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3287852482 0 0) 0x7f11bc12e970 con 0x7f11bc1406c0 2026-02-25T15:01:01.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:01.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:01.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:01.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 10 monclient(hunting): my global_id is 4208 2026-02-25T15:01:01.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1188003610 con 0x7f11bc1406c0 2026-02-25T15:01:01.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2193575869 0 0) 0x7f11bc1258f0 con 0x7f11bc11e790 2026-02-25T15:01:01.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:01.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:01.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:01.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 10 monclient(hunting): my global_id is 4197 2026-02-25T15:01:01.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.435+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11bc12e970 con 0x7f11bc11e790 2026-02-25T15:01:01.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.436+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1666052759 0 0) 0x7f11bc12d750 con 0x7f11bc162250 2026-02-25T15:01:01.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.436+0000 7f11a2ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:01.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.436+0000 7f11a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:01.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.436+0000 7f11a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:01.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.436+0000 7f11a2ffd640 10 monclient(hunting): my global_id is 4201 2026-02-25T15:01:01.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.436+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11bc1258f0 con 0x7f11bc162250 2026-02-25T15:01:01.450 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.438+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3304718059 0 0) 0x7f1188003610 con 0x7f11bc1406c0 2026-02-25T15:01:01.450 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.438+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f11bc12d750 con 0x7f11bc1406c0 2026-02-25T15:01:01.451 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.438+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f11a4003380 con 0x7f11bc1406c0 2026-02-25T15:01:01.451 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.438+0000 7f11a2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:01.451 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.438+0000 7f11a2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:01.451 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.438+0000 7f11a2ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:01.451 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.451 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.451 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.451 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.451 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.451 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.451 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.451 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.451 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.451 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.451 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.438+0000 7f11a2ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.438+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 163627305 0 0) 0x7f11bc12e970 con 0x7f11bc11e790 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.438+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1188003610 con 0x7f11bc11e790 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.438+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f11b0003060 con 0x7f11bc11e790 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.438+0000 7f11a2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.438+0000 7f11a2ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.438+0000 7f11a2ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.452 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3258124097 0 0) 0x7f11bc1258f0 con 0x7f11bc162250 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f11bc12e970 con 0x7f11bc162250 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f11ac003430 con 0x7f11bc162250 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.453 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4133442109 0 0) 0x7f11bc12d750 con 0x7f11bc1406c0 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 >> v1:10.20.193.93:6789/0 conn(0x7f11bc162250 legacy=0x7f11bc161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 >> v1:10.20.193.90:6789/0 conn(0x7f11bc11e790 legacy=0x7f11bc1826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 1 monclient: found mon.c 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc12fc10 con 0x7f11bc1406c0 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11c0c7b640 5 monclient: authenticate success, global_id 4208 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11c0c7b640 10 monclient: _renew_subs 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11c0c7b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11c0c7b640 1 -- 10.20.193.189:0/1966445081 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f11bc12c7a0 con 0x7f11bc1406c0 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11c0c7b640 10 client.4208.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11c0c7b640 10 monclient: _renew_subs 2026-02-25T15:01:01.454 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11c0c7b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11c0c7b640 1 -- 10.20.193.189:0/1966445081 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f11bc12cd90 con 0x7f11bc1406c0 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.439+0000 7f11c0c7b640 1 librados: init done 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.440+0000 7f11c0c7b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.440+0000 7f11c0c7b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.440+0000 7f11c0c7b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.440+0000 7f11c0c7b640 1 -- 10.20.193.189:0/1966445081 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11bc1858b0 con 0x7f11bc1406c0 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.446+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f11a4002f70 con 0x7f11bc1406c0 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.446+0000 7f11a2ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.446+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f11a4003bf0 con 0x7f11bc1406c0 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.446+0000 7f11a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.446+0000 7f11a2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.446+0000 7f11a2ffd640 10 monclient: dump: 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.455 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.456 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.456 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.456 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.456 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.456 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.446+0000 7f11b8d72640 4 set_mon_vals no callback set 2026-02-25T15:01:01.456 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.448+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 3) ==== 103062+0+0 (unknown 2227658051 0 0) 0x7f11a4003bf0 con 0x7f11bc1406c0 2026-02-25T15:01:01.456 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.448+0000 7f11ba575640 10 monclient: get_auth_request con 0x7f1188079ea0 auth_method 0 2026-02-25T15:01:01.456 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.448+0000 7f11a2ffd640 10 client.4208.objecter ms_handle_connect 0x7f1188079ea0 2026-02-25T15:01:01.456 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.448+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(14..14 src has 1..14) ==== 2100+0+0 (unknown 1856864999 0 0) 0x7f11a405ab30 con 0x7f11bc1406c0 2026-02-25T15:01:01.456 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.448+0000 7f11a2ffd640 10 client.4208.objecter ms_dispatch2 0x7f11bc000b90 osd_map(14..14 src has 1..14) 2026-02-25T15:01:01.456 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.448+0000 7f11a2ffd640 3 client.4208.objecter handle_osd_map got epochs [14,14] > 0 2026-02-25T15:01:01.456 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.448+0000 7f11a2ffd640 3 client.4208.objecter handle_osd_map decoding full epoch 14 2026-02-25T15:01:01.456 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.448+0000 7f11a2ffd640 20 client.4208.objecter dump_active .. 0 homeless 2026-02-25T15:01:01.533 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.533+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f11a405f2a0 con 0x7f11bc1406c0 2026-02-25T15:01:01.533 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.533+0000 7f11a2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:01.533 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.533+0000 7f11a2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:01.648 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.647+0000 7f11c0c7b640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "4bad4371-2ca7-4023-8406-0275011eda9c", "id": 12}] 2026-02-25T15:01:01.648 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.647+0000 7f11c0c7b640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "4bad4371-2ca7-4023-8406-0275011eda9c", "id": 12}] 2026-02-25T15:01:01.648 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.647+0000 7f11c0c7b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:01.649 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.647+0000 7f11c0c7b640 1 -- 10.20.193.189:0/1966445081 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd new", "uuid": "4bad4371-2ca7-4023-8406-0275011eda9c", "id": 12} v 0) -- 0x7f11bc12d080 con 0x7f11bc1406c0 2026-02-25T15:01:01.702 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.701+0000 7f11a2ffd640 1 -- 10.20.193.189:0/1966445081 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4bad4371-2ca7-4023-8406-0275011eda9c", "id": 12}]=0 v15) ==== 113+0+3 (unknown 1240960203 0 944643765) 0x7f11a405eba0 con 0x7f11bc1406c0 2026-02-25T15:01:01.702 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.701+0000 7f11a2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "4bad4371-2ca7-4023-8406-0275011eda9c", "id": 12}] 2026-02-25T15:01:01.702 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.701+0000 7f11a2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:01.702 INFO:teuthology.orchestra.run.trial189.stdout:12 2026-02-25T15:01:01.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.703+0000 7f11c0c7b640 10 librados: watch_flush enter 2026-02-25T15:01:01.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.703+0000 7f11c0c7b640 10 librados: watch_flush exit 2026-02-25T15:01:01.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.703+0000 7f11c0c7b640 20 client.4208.objecter shutdown clearing up homeless session... 2026-02-25T15:01:01.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.703+0000 7f11c0c7b640 10 client.4208.objecter successfully canceled tick 2026-02-25T15:01:01.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.703+0000 7f11c0c7b640 1 -- 10.20.193.189:0/1966445081 >> v1:10.20.193.189:6800/3647375982 conn(0x7f1188079ea0 legacy=0x7f118809a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.703+0000 7f11c0c7b640 10 monclient: shutdown 2026-02-25T15:01:01.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.703+0000 7f11c0c7b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:01.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.703+0000 7f11c0c7b640 1 -- 10.20.193.189:0/1966445081 >> v1:10.20.193.94:6789/0 conn(0x7f11bc1406c0 legacy=0x7f11bc124f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.703+0000 7f11c0c7b640 1 -- 10.20.193.189:0/1966445081 shutdown_connections 2026-02-25T15:01:01.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.703+0000 7f11c0c7b640 1 -- 10.20.193.189:0/1966445081 >> 10.20.193.189:0/1966445081 conn(0x7f11bc0588d0 msgr2=0x7f11bc058ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:01.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.703+0000 7f11c0c7b640 1 -- 10.20.193.189:0/1966445081 shutdown_connections 2026-02-25T15:01:01.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.703+0000 7f11c0c7b640 1 -- 10.20.193.189:0/1966445081 wait complete. 2026-02-25T15:01:01.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.703+0000 7f11c0c7b640 1 librados: shutdown 2026-02-25T15:01:01.713 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 5d1ca952-93a5-43f3-9780-8bc85f9ed3d2 13 2026-02-25T15:01:01.781 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.779+0000 7fdd6c302640 10 monclient: get_monmap_and_config 2026-02-25T15:01:01.782 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.779+0000 7fdd6c302640 10 monclient: build_initial_monmap 2026-02-25T15:01:01.782 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.779+0000 7fdd6c302640 10 monclient: monmap: 2026-02-25T15:01:01.782 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:01.782 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.782 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:01:01.781245+0000 2026-02-25T15:01:01.782 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:01:01.781245+0000 2026-02-25T15:01:01.782 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:01.782 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.782 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:01.783 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:01.783 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:01.783 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.783 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.780+0000 7fdd6c302640 1 Processor -- start 2026-02-25T15:01:01.783 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.780+0000 7fdd6c302640 1 -- start start 2026-02-25T15:01:01.783 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.780+0000 7fdd6c302640 10 monclient: init 2026-02-25T15:01:01.783 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.780+0000 7fdd6c302640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:01.783 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.780+0000 7fdd6c302640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:01.783 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.780+0000 7fdd6c302640 10 monclient(hunting): picked mon.noname-a con 0x7fdd6411e790 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:01.783 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.780+0000 7fdd6c302640 10 monclient(hunting): picked mon.noname-c con 0x7fdd641406c0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:01.783 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.780+0000 7fdd6c302640 10 monclient(hunting): picked mon.noname-b con 0x7fdd64162250 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:01.783 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.780+0000 7fdd6c302640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd64183fe0 con 0x7fdd6411e790 2026-02-25T15:01:01.784 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.780+0000 7fdd6c302640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd641851e0 con 0x7fdd64162250 2026-02-25T15:01:01.784 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.780+0000 7fdd6c302640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd641863e0 con 0x7fdd641406c0 2026-02-25T15:01:01.784 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.780+0000 7fdd6c302640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:01.784 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.780+0000 7fdd6c302640 10 monclient(hunting): authenticate will time out at 2969.478598s 2026-02-25T15:01:01.790 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.790+0000 7fdd69876640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fdd641406c0 0x7fdd64160aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.189:32884/0 (socket says 10.20.193.189:32884) 2026-02-25T15:01:01.791 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.790+0000 7fdd69876640 1 -- 10.20.193.189:0/2911674352 learned_addr learned my addr 10.20.193.189:0/2911674352 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:01.819 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.818+0000 7fdd69075640 1 -- 10.20.193.189:0/2911674352 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1962706162 0 0) 0x7fdd641851e0 con 0x7fdd64162250 2026-02-25T15:01:01.819 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.818+0000 7fdd69075640 10 monclient(hunting): _negotiate 2026-02-25T15:01:01.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.818+0000 7fdd69075640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:01.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.818+0000 7fdd69075640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:01.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.818+0000 7fdd69075640 10 monclient(hunting): my global_id is 4204 2026-02-25T15:01:01.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.818+0000 7fdd69075640 1 -- 10.20.193.189:0/2911674352 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd4c003610 con 0x7fdd64162250 2026-02-25T15:01:01.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd69075640 1 -- 10.20.193.189:0/2911674352 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2737063549 0 0) 0x7fdd4c003610 con 0x7fdd64162250 2026-02-25T15:01:01.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd69075640 1 -- 10.20.193.189:0/2911674352 >> v1:10.20.193.94:6789/0 conn(0x7fdd641406c0 legacy=0x7fdd64160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd69075640 1 -- 10.20.193.189:0/2911674352 >> v1:10.20.193.90:6789/0 conn(0x7fdd6411e790 legacy=0x7fdd6411eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:01.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd69075640 10 monclient: _finish_hunting 0 2026-02-25T15:01:01.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd69075640 1 monclient: found mon.noname-b 2026-02-25T15:01:01.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd69075640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:01.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd69075640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:01.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd69075640 1 -- 10.20.193.189:0/2911674352 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd64187680 con 0x7fdd64162250 2026-02-25T15:01:01.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd69075640 10 monclient: _finish_auth 0 2026-02-25T15:01:01.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd69075640 10 monclient: _check_auth_tickets 2026-02-25T15:01:01.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd69075640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:01.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd69075640 1 -- 10.20.193.189:0/2911674352 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fdd60003170 con 0x7fdd64162250 2026-02-25T15:01:01.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd69075640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:01.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd69075640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:01.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd69075640 10 monclient: dump: 2026-02-25T15:01:01.825 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.825 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.825 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.825 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.826 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.826 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.826 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.826 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.826 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.826 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd6c302640 5 monclient: authenticate success, global_id 4204 2026-02-25T15:01:01.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.820+0000 7fdd6c302640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:01.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd69075640 1 -- 10.20.193.189:0/2911674352 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdd600033f0 con 0x7fdd64162250 2026-02-25T15:01:01.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd69075640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:01.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd69075640 1 -- 10.20.193.189:0/2911674352 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fdd60003830 con 0x7fdd64162250 2026-02-25T15:01:01.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd69075640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:01.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd69075640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:01.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd69075640 10 monclient: dump: 2026-02-25T15:01:01.827 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.827 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.827 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.827 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.827 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.827 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.827 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.827 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.827 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.827 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd6c302640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:01.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd6c302640 4 set_mon_vals no callback set 2026-02-25T15:01:01.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd6c302640 10 monclient: shutdown 2026-02-25T15:01:01.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd6c302640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:01.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd6c302640 1 -- 10.20.193.189:0/2911674352 >> v1:10.20.193.93:6789/0 conn(0x7fdd64162250 legacy=0x7fdd641826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd6c302640 1 -- 10.20.193.189:0/2911674352 shutdown_connections 2026-02-25T15:01:01.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd6c302640 1 -- 10.20.193.189:0/2911674352 >> 10.20.193.189:0/2911674352 conn(0x7fdd640588d0 msgr2=0x7fdd64058cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:01.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd6c302640 1 -- 10.20.193.189:0/2911674352 shutdown_connections 2026-02-25T15:01:01.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd6c302640 1 -- 10.20.193.189:0/2911674352 wait complete. 2026-02-25T15:01:01.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd6c302640 10 monclient: build_initial_monmap 2026-02-25T15:01:01.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd6c302640 10 monclient: monmap: 2026-02-25T15:01:01.828 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:01.828 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:01.829 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:01:01.829 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:01:01.829 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:01.829 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:01:01.829 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:01.829 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:01.829 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:01.829 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd6c302640 1 librados: starting msgr at 2026-02-25T15:01:01.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd6c302640 1 librados: starting objecter 2026-02-25T15:01:01.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd6c302640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:01.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.821+0000 7fdd6c302640 1 Processor -- start 2026-02-25T15:01:01.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6c302640 1 -- start start 2026-02-25T15:01:01.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6c302640 1 librados: setting wanted keys 2026-02-25T15:01:01.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6c302640 1 librados: calling monclient init 2026-02-25T15:01:01.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6c302640 10 monclient: init 2026-02-25T15:01:01.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6c302640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:01.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6c302640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:01.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6c302640 10 monclient(hunting): picked mon.noname-c con 0x7fdd6411e790 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:01.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6c302640 10 monclient(hunting): picked mon.noname-a con 0x7fdd641406c0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:01.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6c302640 10 monclient(hunting): picked mon.noname-b con 0x7fdd64162250 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:01.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6c302640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd64125770 con 0x7fdd641406c0 2026-02-25T15:01:01.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6c302640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd6412d5b0 con 0x7fdd64162250 2026-02-25T15:01:01.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6c302640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd6412e7d0 con 0x7fdd6411e790 2026-02-25T15:01:01.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6c302640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:01.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6c302640 10 monclient(hunting): authenticate will time out at 2969.520244s 2026-02-25T15:01:01.831 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6a077640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fdd6411e790 0x7fdd641826c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.189:32892/0 (socket says 10.20.193.189:32892) 2026-02-25T15:01:01.831 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd6a077640 1 -- 10.20.193.189:0/3871291809 learned_addr learned my addr 10.20.193.189:0/3871291809 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:01.831 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd4affd640 10 client.?.objecter ms_handle_connect 0x7fdd64162250 2026-02-25T15:01:01.831 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd4affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:01.831 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd4affd640 10 client.?.objecter ms_handle_connect 0x7fdd6411e790 2026-02-25T15:01:01.831 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.822+0000 7fdd4affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:01.833 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.823+0000 7fdd4affd640 1 -- 10.20.193.189:0/3871291809 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4217807980 0 0) 0x7fdd6412d5b0 con 0x7fdd64162250 2026-02-25T15:01:01.833 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.823+0000 7fdd4affd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:01.834 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.823+0000 7fdd4affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:01.834 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.823+0000 7fdd4affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:01.834 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.823+0000 7fdd4affd640 10 monclient(hunting): my global_id is 4207 2026-02-25T15:01:01.834 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.823+0000 7fdd4affd640 1 -- 10.20.193.189:0/3871291809 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd34003610 con 0x7fdd64162250 2026-02-25T15:01:01.834 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.826+0000 7fdd4affd640 1 -- 10.20.193.189:0/3871291809 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2363211309 0 0) 0x7fdd34003610 con 0x7fdd64162250 2026-02-25T15:01:01.834 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.826+0000 7fdd4affd640 1 -- 10.20.193.189:0/3871291809 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdd6412d5b0 con 0x7fdd64162250 2026-02-25T15:01:01.834 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.826+0000 7fdd4affd640 1 -- 10.20.193.189:0/3871291809 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fdd60003080 con 0x7fdd64162250 2026-02-25T15:01:01.834 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.826+0000 7fdd4affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:01.834 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.826+0000 7fdd4affd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:01.834 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.826+0000 7fdd4affd640 10 monclient(hunting): dump: 2026-02-25T15:01:01.834 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.834 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.835 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.835 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.835 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.835 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.835 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.835 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.835 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.835 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.835 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.826+0000 7fdd4affd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:01.835 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd4affd640 1 -- 10.20.193.189:0/3871291809 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2042462814 0 0) 0x7fdd6412d5b0 con 0x7fdd64162250 2026-02-25T15:01:01.835 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd4affd640 1 -- 10.20.193.189:0/3871291809 >> v1:10.20.193.94:6789/0 conn(0x7fdd6411e790 legacy=0x7fdd641826c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.835 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd4affd640 1 -- 10.20.193.189:0/3871291809 >> v1:10.20.193.90:6789/0 conn(0x7fdd641406c0 legacy=0x7fdd64124d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:01.836 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd4affd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:01.836 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd4affd640 1 monclient: found mon.b 2026-02-25T15:01:01.836 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd4affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:01.836 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd4affd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:01.836 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd4affd640 1 -- 10.20.193.189:0/3871291809 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6412fa70 con 0x7fdd64162250 2026-02-25T15:01:01.836 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd4affd640 10 monclient: _finish_auth 0 2026-02-25T15:01:01.836 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd4affd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:01.836 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd4affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:01.836 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd6c302640 5 monclient: authenticate success, global_id 4207 2026-02-25T15:01:01.836 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd6c302640 10 monclient: _renew_subs 2026-02-25T15:01:01.836 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd6c302640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:01.836 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd6c302640 1 -- 10.20.193.189:0/3871291809 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6412ea00 con 0x7fdd64162250 2026-02-25T15:01:01.836 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd6c302640 10 client.4207.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:01.837 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd6c302640 10 monclient: _renew_subs 2026-02-25T15:01:01.837 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd6c302640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:01.837 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd6c302640 1 -- 10.20.193.189:0/3871291809 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fdd6412f040 con 0x7fdd64162250 2026-02-25T15:01:01.837 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd6c302640 1 librados: init done 2026-02-25T15:01:01.837 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd6c302640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:01.837 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd6c302640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:01.837 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd6c302640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:01.837 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.828+0000 7fdd6c302640 1 -- 10.20.193.189:0/3871291809 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd641858e0 con 0x7fdd64162250 2026-02-25T15:01:01.838 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.833+0000 7fdd4affd640 1 -- 10.20.193.189:0/3871291809 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdd60003350 con 0x7fdd64162250 2026-02-25T15:01:01.838 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.833+0000 7fdd4affd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:01.838 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.833+0000 7fdd68874640 4 set_mon_vals no callback set 2026-02-25T15:01:01.838 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.837+0000 7fdd4affd640 1 -- 10.20.193.189:0/3871291809 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fdd60003f70 con 0x7fdd64162250 2026-02-25T15:01:01.838 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.837+0000 7fdd4affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:01.838 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.837+0000 7fdd4affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:01.838 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.837+0000 7fdd4affd640 10 monclient: dump: 2026-02-25T15:01:01.839 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:01.839 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:01.839 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.839 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:01.839 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:01.839 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:01.839 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:01.839 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:01.839 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:01.839 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:01.840 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.838+0000 7fdd4affd640 1 -- 10.20.193.189:0/3871291809 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 4) ==== 103168+0+0 (unknown 1070002493 0 0) 0x7fdd6001d420 con 0x7fdd64162250 2026-02-25T15:01:01.840 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.838+0000 7fdd6a077640 10 monclient: get_auth_request con 0x7fdd34079cb0 auth_method 0 2026-02-25T15:01:01.840 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.839+0000 7fdd4affd640 10 client.4207.objecter ms_handle_connect 0x7fdd34079cb0 2026-02-25T15:01:01.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.839+0000 7fdd4affd640 1 -- 10.20.193.189:0/3871291809 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(15..15 src has 1..15) ==== 2215+0+0 (unknown 744762936 0 0) 0x7fdd6005a040 con 0x7fdd64162250 2026-02-25T15:01:01.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.839+0000 7fdd4affd640 10 client.4207.objecter ms_dispatch2 0x7fdd64000b90 osd_map(15..15 src has 1..15) 2026-02-25T15:01:01.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.839+0000 7fdd4affd640 3 client.4207.objecter handle_osd_map got epochs [15,15] > 0 2026-02-25T15:01:01.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.839+0000 7fdd4affd640 3 client.4207.objecter handle_osd_map decoding full epoch 15 2026-02-25T15:01:01.841 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.839+0000 7fdd4affd640 20 client.4207.objecter dump_active .. 0 homeless 2026-02-25T15:01:01.922 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.921+0000 7fdd4affd640 1 -- 10.20.193.189:0/3871291809 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fdd6005e7b0 con 0x7fdd64162250 2026-02-25T15:01:01.922 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.921+0000 7fdd4affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:01.923 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:01.921+0000 7fdd4affd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:02.034 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.032+0000 7fdd6c302640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "5d1ca952-93a5-43f3-9780-8bc85f9ed3d2", "id": 13}] 2026-02-25T15:01:02.034 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.032+0000 7fdd6c302640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "5d1ca952-93a5-43f3-9780-8bc85f9ed3d2", "id": 13}] 2026-02-25T15:01:02.034 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.032+0000 7fdd6c302640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:02.034 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.032+0000 7fdd6c302640 1 -- 10.20.193.189:0/3871291809 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd new", "uuid": "5d1ca952-93a5-43f3-9780-8bc85f9ed3d2", "id": 13} v 0) -- 0x7fdd6412f330 con 0x7fdd64162250 2026-02-25T15:01:02.081 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.081+0000 7fdd4affd640 1 -- 10.20.193.189:0/3871291809 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5d1ca952-93a5-43f3-9780-8bc85f9ed3d2", "id": 13}]=0 v16) ==== 113+0+3 (unknown 4149743466 0 736921282) 0x7fdd6005e0b0 con 0x7fdd64162250 2026-02-25T15:01:02.081 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.081+0000 7fdd4affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "5d1ca952-93a5-43f3-9780-8bc85f9ed3d2", "id": 13}] 2026-02-25T15:01:02.081 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.081+0000 7fdd4affd640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:02.082 INFO:teuthology.orchestra.run.trial189.stdout:13 2026-02-25T15:01:02.083 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.081+0000 7fdd6c302640 10 librados: watch_flush enter 2026-02-25T15:01:02.083 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.081+0000 7fdd6c302640 10 librados: watch_flush exit 2026-02-25T15:01:02.083 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.081+0000 7fdd6c302640 20 client.4207.objecter shutdown clearing up homeless session... 2026-02-25T15:01:02.083 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.081+0000 7fdd6c302640 10 client.4207.objecter successfully canceled tick 2026-02-25T15:01:02.084 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.081+0000 7fdd6c302640 1 -- 10.20.193.189:0/3871291809 >> v1:10.20.193.189:6800/3647375982 conn(0x7fdd34079cb0 legacy=0x7fdd3409a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.084 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.081+0000 7fdd6c302640 10 monclient: shutdown 2026-02-25T15:01:02.084 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.081+0000 7fdd6c302640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:02.084 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.081+0000 7fdd6c302640 1 -- 10.20.193.189:0/3871291809 >> v1:10.20.193.93:6789/0 conn(0x7fdd64162250 legacy=0x7fdd64161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.084 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.082+0000 7fdd6c302640 1 -- 10.20.193.189:0/3871291809 shutdown_connections 2026-02-25T15:01:02.084 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.082+0000 7fdd6c302640 1 -- 10.20.193.189:0/3871291809 >> 10.20.193.189:0/3871291809 conn(0x7fdd640588d0 msgr2=0x7fdd64184410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:02.084 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.082+0000 7fdd6c302640 1 -- 10.20.193.189:0/3871291809 shutdown_connections 2026-02-25T15:01:02.084 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.082+0000 7fdd6c302640 1 -- 10.20.193.189:0/3871291809 wait complete. 2026-02-25T15:01:02.084 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.082+0000 7fdd6c302640 1 librados: shutdown 2026-02-25T15:01:02.094 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 50853d20-2756-4720-b7aa-3020ded29189 14 2026-02-25T15:01:02.159 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 10 monclient: get_monmap_and_config 2026-02-25T15:01:02.159 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 10 monclient: build_initial_monmap 2026-02-25T15:01:02.159 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 10 monclient: monmap: 2026-02-25T15:01:02.159 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:02.159 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.159 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:01:02.158317+0000 2026-02-25T15:01:02.160 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:01:02.158317+0000 2026-02-25T15:01:02.160 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:02.160 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:02.160 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:02.160 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:02.160 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:02.160 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.160 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 1 Processor -- start 2026-02-25T15:01:02.160 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 1 -- start start 2026-02-25T15:01:02.160 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 10 monclient: init 2026-02-25T15:01:02.160 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:02.160 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:02.161 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 10 monclient(hunting): picked mon.noname-a con 0x7f15e811e790 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:02.161 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 10 monclient(hunting): picked mon.noname-c con 0x7f15e81406c0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:02.161 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 10 monclient(hunting): picked mon.noname-b con 0x7f15e8162250 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:02.161 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e8183fe0 con 0x7f15e811e790 2026-02-25T15:01:02.161 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e81851e0 con 0x7f15e8162250 2026-02-25T15:01:02.161 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e81863e0 con 0x7f15e81406c0 2026-02-25T15:01:02.161 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:02.161 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.157+0000 7f15ed29f640 10 monclient(hunting): authenticate will time out at 2969.855657s 2026-02-25T15:01:02.161 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.158+0000 7f15e67fc640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f15e81406c0 0x7f15e8160aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.189:32896/0 (socket says 10.20.193.189:32896) 2026-02-25T15:01:02.161 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.158+0000 7f15e67fc640 1 -- 10.20.193.189:0/1649332926 learned_addr learned my addr 10.20.193.189:0/1649332926 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:02.167 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 1 -- 10.20.193.189:0/1649332926 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3614221987 0 0) 0x7f15e8183fe0 con 0x7f15e811e790 2026-02-25T15:01:02.167 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.167 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.167 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.167 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 10 monclient(hunting): my global_id is 4200 2026-02-25T15:01:02.167 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 1 -- 10.20.193.189:0/1649332926 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15d4003610 con 0x7f15e811e790 2026-02-25T15:01:02.167 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 1 -- 10.20.193.189:0/1649332926 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1420283073 0 0) 0x7f15e81851e0 con 0x7f15e8162250 2026-02-25T15:01:02.167 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.168 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.168 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.168 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 10 monclient(hunting): my global_id is 4210 2026-02-25T15:01:02.168 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 1 -- 10.20.193.189:0/1649332926 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15e8183fe0 con 0x7f15e8162250 2026-02-25T15:01:02.168 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 1 -- 10.20.193.189:0/1649332926 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1272345492 0 0) 0x7f15e81863e0 con 0x7f15e81406c0 2026-02-25T15:01:02.168 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.168 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.168 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.168 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 10 monclient(hunting): my global_id is 4211 2026-02-25T15:01:02.168 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.159+0000 7f15e5ffb640 1 -- 10.20.193.189:0/1649332926 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15e81851e0 con 0x7f15e81406c0 2026-02-25T15:01:02.168 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15e5ffb640 1 -- 10.20.193.189:0/1649332926 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1457746252 0 0) 0x7f15e8183fe0 con 0x7f15e8162250 2026-02-25T15:01:02.169 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15e5ffb640 1 -- 10.20.193.189:0/1649332926 >> v1:10.20.193.94:6789/0 conn(0x7f15e81406c0 legacy=0x7f15e8160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.169 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15e5ffb640 1 -- 10.20.193.189:0/1649332926 >> v1:10.20.193.90:6789/0 conn(0x7f15e811e790 legacy=0x7f15e811eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.169 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15e5ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:02.169 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15e5ffb640 1 monclient: found mon.noname-b 2026-02-25T15:01:02.169 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15e5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:02.169 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15e5ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:02.169 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15e5ffb640 1 -- 10.20.193.189:0/1649332926 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e8187680 con 0x7f15e8162250 2026-02-25T15:01:02.169 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15e5ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:02.169 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15e5ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:02.169 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15e5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:02.169 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15e5ffb640 1 -- 10.20.193.189:0/1649332926 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f15d8003170 con 0x7f15e8162250 2026-02-25T15:01:02.169 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15ed29f640 5 monclient: authenticate success, global_id 4210 2026-02-25T15:01:02.170 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15ed29f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:02.170 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:02.170 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15e5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:02.170 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15e5ffb640 10 monclient: dump: 2026-02-25T15:01:02.170 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:02.170 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.170 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.170 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.170 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.170 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:02.170 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.170 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.170 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.171 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.171 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.161+0000 7f15ed29f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:02.171 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15e5ffb640 1 -- 10.20.193.189:0/1649332926 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f15d80033b0 con 0x7f15e8162250 2026-02-25T15:01:02.171 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15e5ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:02.171 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15e5ffb640 1 -- 10.20.193.189:0/1649332926 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f15d80037f0 con 0x7f15e8162250 2026-02-25T15:01:02.171 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:02.171 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 4 set_mon_vals no callback set 2026-02-25T15:01:02.171 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:02.171 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15e5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:02.171 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15e5ffb640 10 monclient: dump: 2026-02-25T15:01:02.171 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:02.172 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.172 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.172 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.172 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.172 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:02.172 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.172 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.172 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.172 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.172 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 10 monclient: shutdown 2026-02-25T15:01:02.172 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:02.172 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 1 -- 10.20.193.189:0/1649332926 >> v1:10.20.193.93:6789/0 conn(0x7f15e8162250 legacy=0x7f15e81826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.173 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 1 -- 10.20.193.189:0/1649332926 shutdown_connections 2026-02-25T15:01:02.173 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 1 -- 10.20.193.189:0/1649332926 >> 10.20.193.189:0/1649332926 conn(0x7f15e80588d0 msgr2=0x7f15e8058cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:02.173 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 1 -- 10.20.193.189:0/1649332926 shutdown_connections 2026-02-25T15:01:02.173 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 1 -- 10.20.193.189:0/1649332926 wait complete. 2026-02-25T15:01:02.173 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 10 monclient: build_initial_monmap 2026-02-25T15:01:02.173 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 10 monclient: monmap: 2026-02-25T15:01:02.173 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:02.173 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:02.173 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:01:02.173 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:01:02.173 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:02.173 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:01:02.173 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:02.174 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:02.174 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:02.174 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.174 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 1 librados: starting msgr at 2026-02-25T15:01:02.174 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 1 librados: starting objecter 2026-02-25T15:01:02.174 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:02.174 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 1 Processor -- start 2026-02-25T15:01:02.174 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 1 -- start start 2026-02-25T15:01:02.174 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 1 librados: setting wanted keys 2026-02-25T15:01:02.174 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 1 librados: calling monclient init 2026-02-25T15:01:02.174 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 10 monclient: init 2026-02-25T15:01:02.174 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:02.174 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:02.175 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.162+0000 7f15ed29f640 10 monclient(hunting): picked mon.noname-b con 0x7f15e811e790 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:02.175 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.163+0000 7f15ed29f640 10 monclient(hunting): picked mon.noname-c con 0x7f15e81406c0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:02.175 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.163+0000 7f15ed29f640 10 monclient(hunting): picked mon.noname-a con 0x7f15e8162250 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:02.175 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.163+0000 7f15ed29f640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e8104730 con 0x7f15e8162250 2026-02-25T15:01:02.175 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.163+0000 7f15ed29f640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e810c570 con 0x7f15e811e790 2026-02-25T15:01:02.175 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.163+0000 7f15ed29f640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e810d790 con 0x7f15e81406c0 2026-02-25T15:01:02.175 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.163+0000 7f15ed29f640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:02.175 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.163+0000 7f15ed29f640 10 monclient(hunting): authenticate will time out at 2969.861035s 2026-02-25T15:01:02.175 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.163+0000 7f15e67fc640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f15e81406c0 0x7f15e8103d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.189:32910/0 (socket says 10.20.193.189:32910) 2026-02-25T15:01:02.175 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.163+0000 7f15e67fc640 1 -- 10.20.193.189:0/3629872134 learned_addr learned my addr 10.20.193.189:0/3629872134 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:02.175 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.163+0000 7f15c77fe640 10 client.?.objecter ms_handle_connect 0x7f15e81406c0 2026-02-25T15:01:02.175 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.163+0000 7f15c77fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:02.176 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.163+0000 7f15c77fe640 10 client.?.objecter ms_handle_connect 0x7f15e811e790 2026-02-25T15:01:02.176 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.163+0000 7f15c77fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:02.176 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.163+0000 7f15c77fe640 10 client.?.objecter ms_handle_connect 0x7f15e8162250 2026-02-25T15:01:02.176 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.163+0000 7f15c77fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:02.176 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 507755871 0 0) 0x7f15e8187680 con 0x7f15e8162250 2026-02-25T15:01:02.176 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.176 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.176 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.176 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 10 monclient(hunting): my global_id is 4203 2026-02-25T15:01:02.176 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15b4003610 con 0x7f15e8162250 2026-02-25T15:01:02.176 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4196045077 0 0) 0x7f15e810c570 con 0x7f15e811e790 2026-02-25T15:01:02.176 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.176 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.177 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.177 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 10 monclient(hunting): my global_id is 4213 2026-02-25T15:01:02.177 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15e8187680 con 0x7f15e811e790 2026-02-25T15:01:02.177 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2131018473 0 0) 0x7f15e810d790 con 0x7f15e81406c0 2026-02-25T15:01:02.177 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.177 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.177 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.177 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 10 monclient(hunting): my global_id is 4214 2026-02-25T15:01:02.177 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.164+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15e810c570 con 0x7f15e81406c0 2026-02-25T15:01:02.180 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3019493021 0 0) 0x7f15e810c570 con 0x7f15e81406c0 2026-02-25T15:01:02.180 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f15e810d790 con 0x7f15e81406c0 2026-02-25T15:01:02.180 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f15d0003060 con 0x7f15e81406c0 2026-02-25T15:01:02.180 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:02.180 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:02.180 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 10 monclient(hunting): dump: 2026-02-25T15:01:02.180 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:02.180 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.180 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.180 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.180 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.181 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:02.181 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.181 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.181 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.181 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.181 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:02.181 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1055299483 0 0) 0x7f15e8187680 con 0x7f15e811e790 2026-02-25T15:01:02.181 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f15e810c570 con 0x7f15e811e790 2026-02-25T15:01:02.181 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f15dc0030b0 con 0x7f15e811e790 2026-02-25T15:01:02.181 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:02.181 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:02.181 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 10 monclient(hunting): dump: 2026-02-25T15:01:02.181 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:02.181 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.181 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2452797363 0 0) 0x7f15b4003610 con 0x7f15e8162250 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f15e8187680 con 0x7f15e8162250 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f15d8003700 con 0x7f15e8162250 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.167+0000 7f15c77fe640 10 monclient(hunting): dump: 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.182 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 481082224 0 0) 0x7f15e810d790 con 0x7f15e81406c0 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 >> v1:10.20.193.93:6789/0 conn(0x7f15e811e790 legacy=0x7f15e81826c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 >> v1:10.20.193.90:6789/0 conn(0x7f15e8162250 legacy=0x7f15e8161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15c77fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15c77fe640 1 monclient: found mon.c 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15c77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15c77fe640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e810ea30 con 0x7f15e81406c0 2026-02-25T15:01:02.183 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15c77fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15c77fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15ed29f640 5 monclient: authenticate success, global_id 4214 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15ed29f640 10 monclient: _renew_subs 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15ed29f640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15ed29f640 1 -- 10.20.193.189:0/3629872134 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f15e810c7a0 con 0x7f15e81406c0 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15ed29f640 10 client.4214.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15ed29f640 10 monclient: _renew_subs 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15ed29f640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15ed29f640 1 -- 10.20.193.189:0/3629872134 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f15e810cde0 con 0x7f15e81406c0 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15ed29f640 1 librados: init done 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15ed29f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15ed29f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15ed29f640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.169+0000 7f15ed29f640 1 -- 10.20.193.189:0/3629872134 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e81846b0 con 0x7f15e81406c0 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.176+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f15d00048d0 con 0x7f15e81406c0 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.176+0000 7f15c77fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:02.184 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.176+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f15d0004ca0 con 0x7f15e81406c0 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.176+0000 7f15c77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.176+0000 7f15c77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.176+0000 7f15e57fa640 4 set_mon_vals no callback set 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.176+0000 7f15c77fe640 10 monclient: dump: 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.177+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 4) ==== 103168+0+0 (unknown 1070002493 0 0) 0x7f15d0004ca0 con 0x7f15e81406c0 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.178+0000 7f15e6ffd640 10 monclient: get_auth_request con 0x7f15b4079fb0 auth_method 0 2026-02-25T15:01:02.185 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.178+0000 7f15c77fe640 10 client.4214.objecter ms_handle_connect 0x7f15b4079fb0 2026-02-25T15:01:02.186 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.178+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(16..16 src has 1..16) ==== 2330+0+0 (unknown 4084678258 0 0) 0x7f15d005acf0 con 0x7f15e81406c0 2026-02-25T15:01:02.186 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.178+0000 7f15c77fe640 10 client.4214.objecter ms_dispatch2 0x7f15e8000b90 osd_map(16..16 src has 1..16) 2026-02-25T15:01:02.186 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.178+0000 7f15c77fe640 3 client.4214.objecter handle_osd_map got epochs [16,16] > 0 2026-02-25T15:01:02.186 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.178+0000 7f15c77fe640 3 client.4214.objecter handle_osd_map decoding full epoch 16 2026-02-25T15:01:02.186 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.178+0000 7f15c77fe640 20 client.4214.objecter dump_active .. 0 homeless 2026-02-25T15:01:02.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.260+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f15d005f460 con 0x7f15e81406c0 2026-02-25T15:01:02.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.260+0000 7f15c77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:02.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.260+0000 7f15c77fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:02.373 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.371+0000 7f15ed29f640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "50853d20-2756-4720-b7aa-3020ded29189", "id": 14}] 2026-02-25T15:01:02.373 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.371+0000 7f15ed29f640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "50853d20-2756-4720-b7aa-3020ded29189", "id": 14}] 2026-02-25T15:01:02.373 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.371+0000 7f15ed29f640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:02.373 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.371+0000 7f15ed29f640 1 -- 10.20.193.189:0/3629872134 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd new", "uuid": "50853d20-2756-4720-b7aa-3020ded29189", "id": 14} v 0) -- 0x7f15e810d0d0 con 0x7f15e81406c0 2026-02-25T15:01:02.419 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.419+0000 7f15c77fe640 1 -- 10.20.193.189:0/3629872134 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "50853d20-2756-4720-b7aa-3020ded29189", "id": 14}]=0 v17) ==== 113+0+3 (unknown 913136240 0 1367428999) 0x7f15d005ed60 con 0x7f15e81406c0 2026-02-25T15:01:02.420 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.419+0000 7f15c77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "50853d20-2756-4720-b7aa-3020ded29189", "id": 14}] 2026-02-25T15:01:02.420 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.419+0000 7f15c77fe640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:02.420 INFO:teuthology.orchestra.run.trial189.stdout:14 2026-02-25T15:01:02.421 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.419+0000 7f15ed29f640 10 librados: watch_flush enter 2026-02-25T15:01:02.422 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.419+0000 7f15ed29f640 10 librados: watch_flush exit 2026-02-25T15:01:02.422 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.419+0000 7f15ed29f640 20 client.4214.objecter shutdown clearing up homeless session... 2026-02-25T15:01:02.422 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.419+0000 7f15ed29f640 10 client.4214.objecter successfully canceled tick 2026-02-25T15:01:02.422 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.419+0000 7f15ed29f640 1 -- 10.20.193.189:0/3629872134 >> v1:10.20.193.189:6800/3647375982 conn(0x7f15b4079fb0 legacy=0x7f15b409a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.422 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.419+0000 7f15ed29f640 10 monclient: shutdown 2026-02-25T15:01:02.422 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.419+0000 7f15ed29f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:02.422 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.419+0000 7f15ed29f640 1 -- 10.20.193.189:0/3629872134 >> v1:10.20.193.94:6789/0 conn(0x7f15e81406c0 legacy=0x7f15e8103d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.422 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.419+0000 7f15ed29f640 1 -- 10.20.193.189:0/3629872134 shutdown_connections 2026-02-25T15:01:02.422 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.419+0000 7f15ed29f640 1 -- 10.20.193.189:0/3629872134 >> 10.20.193.189:0/3629872134 conn(0x7f15e80588d0 msgr2=0x7f15e81833b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:02.422 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.420+0000 7f15ed29f640 1 -- 10.20.193.189:0/3629872134 shutdown_connections 2026-02-25T15:01:02.422 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.420+0000 7f15ed29f640 1 -- 10.20.193.189:0/3629872134 wait complete. 2026-02-25T15:01:02.423 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.420+0000 7f15ed29f640 1 librados: shutdown 2026-02-25T15:01:02.430 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 4445eda7-fb77-4f2c-8fda-77667df22b47 15 2026-02-25T15:01:02.495 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 10 monclient: get_monmap_and_config 2026-02-25T15:01:02.496 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 10 monclient: build_initial_monmap 2026-02-25T15:01:02.496 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 10 monclient: monmap: 2026-02-25T15:01:02.496 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:02.496 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.496 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:01:02.494495+0000 2026-02-25T15:01:02.496 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:01:02.494495+0000 2026-02-25T15:01:02.496 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:02.497 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:02.497 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:02.497 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:02.497 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:02.497 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.497 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 1 Processor -- start 2026-02-25T15:01:02.497 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 1 -- start start 2026-02-25T15:01:02.497 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 10 monclient: init 2026-02-25T15:01:02.497 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:02.497 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:02.497 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 10 monclient(hunting): picked mon.noname-a con 0x7f4ef811c500 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:02.498 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 10 monclient(hunting): picked mon.noname-c con 0x7f4ef811e420 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:02.498 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 10 monclient(hunting): picked mon.noname-b con 0x7f4ef811ffc0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:02.498 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ef8120d10 con 0x7f4ef811c500 2026-02-25T15:01:02.498 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ef8120f00 con 0x7f4ef811ffc0 2026-02-25T15:01:02.498 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ef81210f0 con 0x7f4ef811e420 2026-02-25T15:01:02.498 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:02.498 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.493+0000 7f4eff109640 10 monclient(hunting): authenticate will time out at 2970.191822s 2026-02-25T15:01:02.498 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.494+0000 7f4efd67f640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f4ef811ffc0 0x7f4ef8120420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.189:41766/0 (socket says 10.20.193.189:41766) 2026-02-25T15:01:02.498 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.494+0000 7f4efd67f640 1 -- 10.20.193.189:0/2025078320 learned_addr learned my addr 10.20.193.189:0/2025078320 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:02.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 1 -- 10.20.193.189:0/2025078320 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3283857790 0 0) 0x7f4ef8120f00 con 0x7f4ef811ffc0 2026-02-25T15:01:02.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 10 monclient(hunting): my global_id is 4216 2026-02-25T15:01:02.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 1 -- 10.20.193.189:0/2025078320 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ee4003610 con 0x7f4ef811ffc0 2026-02-25T15:01:02.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 1 -- 10.20.193.189:0/2025078320 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1631221066 0 0) 0x7f4ef81210f0 con 0x7f4ef811e420 2026-02-25T15:01:02.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 10 monclient(hunting): my global_id is 4217 2026-02-25T15:01:02.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 1 -- 10.20.193.189:0/2025078320 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ef8120f00 con 0x7f4ef811e420 2026-02-25T15:01:02.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 1 -- 10.20.193.189:0/2025078320 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3292683940 0 0) 0x7f4ef8120d10 con 0x7f4ef811c500 2026-02-25T15:01:02.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 10 monclient(hunting): my global_id is 4206 2026-02-25T15:01:02.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.495+0000 7f4eef7fe640 1 -- 10.20.193.189:0/2025078320 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ef81210f0 con 0x7f4ef811c500 2026-02-25T15:01:02.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eef7fe640 1 -- 10.20.193.189:0/2025078320 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3513989050 0 0) 0x7f4ef8120f00 con 0x7f4ef811e420 2026-02-25T15:01:02.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eef7fe640 1 -- 10.20.193.189:0/2025078320 >> v1:10.20.193.93:6789/0 conn(0x7f4ef811ffc0 legacy=0x7f4ef8120420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eef7fe640 1 -- 10.20.193.189:0/2025078320 >> v1:10.20.193.90:6789/0 conn(0x7f4ef811c500 legacy=0x7f4ef811c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eef7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:02.509 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eef7fe640 1 monclient: found mon.noname-c 2026-02-25T15:01:02.509 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:02.509 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eef7fe640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:02.509 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eef7fe640 1 -- 10.20.193.189:0/2025078320 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef8121380 con 0x7f4ef811e420 2026-02-25T15:01:02.509 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eef7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:02.509 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eef7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:02.509 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:02.509 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eef7fe640 1 -- 10.20.193.189:0/2025078320 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4ee0002df0 con 0x7f4ef811e420 2026-02-25T15:01:02.509 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eff109640 5 monclient: authenticate success, global_id 4217 2026-02-25T15:01:02.509 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eff109640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:02.509 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eef7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:02.509 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eef7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:02.510 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eef7fe640 10 monclient: dump: 2026-02-25T15:01:02.510 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:02.510 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.510 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.510 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.510 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.510 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:02.510 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.510 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.510 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.510 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.510 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.497+0000 7f4eff109640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:02.511 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eef7fe640 1 -- 10.20.193.189:0/2025078320 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4ee0003070 con 0x7f4ef811e420 2026-02-25T15:01:02.511 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eef7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:02.511 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eef7fe640 1 -- 10.20.193.189:0/2025078320 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4ee0003470 con 0x7f4ef811e420 2026-02-25T15:01:02.511 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eef7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:02.511 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eef7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:02.511 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eef7fe640 10 monclient: dump: 2026-02-25T15:01:02.511 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:02.511 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.511 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.511 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.511 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.511 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:02.512 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.512 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.512 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.512 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.512 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:02.512 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 4 set_mon_vals no callback set 2026-02-25T15:01:02.512 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 10 monclient: shutdown 2026-02-25T15:01:02.512 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:02.512 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 1 -- 10.20.193.189:0/2025078320 >> v1:10.20.193.94:6789/0 conn(0x7f4ef811e420 legacy=0x7f4ef811e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.512 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 1 -- 10.20.193.189:0/2025078320 shutdown_connections 2026-02-25T15:01:02.512 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 1 -- 10.20.193.189:0/2025078320 >> 10.20.193.189:0/2025078320 conn(0x7f4ef80588d0 msgr2=0x7f4ef8058cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:02.512 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 1 -- 10.20.193.189:0/2025078320 shutdown_connections 2026-02-25T15:01:02.512 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 1 -- 10.20.193.189:0/2025078320 wait complete. 2026-02-25T15:01:02.513 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 10 monclient: build_initial_monmap 2026-02-25T15:01:02.513 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 10 monclient: monmap: 2026-02-25T15:01:02.513 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-25T15:01:02.513 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:02.513 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-25T15:01:02.513 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-25T15:01:02.513 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:02.513 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-25T15:01:02.513 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:02.513 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:02.513 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:02.513 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.514 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 1 librados: starting msgr at 2026-02-25T15:01:02.514 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 1 librados: starting objecter 2026-02-25T15:01:02.514 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:02.514 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 1 Processor -- start 2026-02-25T15:01:02.514 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 1 -- start start 2026-02-25T15:01:02.514 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 1 librados: setting wanted keys 2026-02-25T15:01:02.514 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 1 librados: calling monclient init 2026-02-25T15:01:02.514 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.498+0000 7f4eff109640 10 monclient: init 2026-02-25T15:01:02.514 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eff109640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:02.514 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eff109640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:02.514 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eff109640 10 monclient(hunting): picked mon.noname-b con 0x7f4ef811c500 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:02.514 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eff109640 10 monclient(hunting): picked mon.noname-a con 0x7f4ef811e420 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:02.514 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eff109640 10 monclient(hunting): picked mon.noname-c con 0x7f4ef811ffc0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:02.515 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eff109640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ef8179630 con 0x7f4ef811e420 2026-02-25T15:01:02.515 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eff109640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ef817a810 con 0x7f4ef811c500 2026-02-25T15:01:02.515 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eff109640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ef817b9f0 con 0x7f4ef811ffc0 2026-02-25T15:01:02.515 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eff109640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:02.515 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eff109640 10 monclient(hunting): authenticate will time out at 2970.197141s 2026-02-25T15:01:02.515 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4efce7e640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f4ef811c500 0x7f4ef812fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.189:41782/0 (socket says 10.20.193.189:41782) 2026-02-25T15:01:02.515 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4efce7e640 1 -- 10.20.193.189:0/257046860 learned_addr learned my addr 10.20.193.189:0/257046860 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:01:02.515 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eed7fa640 10 client.?.objecter ms_handle_connect 0x7f4ef811e420 2026-02-25T15:01:02.515 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eed7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:02.515 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eed7fa640 10 client.?.objecter ms_handle_connect 0x7f4ef811c500 2026-02-25T15:01:02.515 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eed7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:02.515 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eed7fa640 10 client.?.objecter ms_handle_connect 0x7f4ef811ffc0 2026-02-25T15:01:02.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.499+0000 7f4eed7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:02.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2504083533 0 0) 0x7f4ef8179630 con 0x7f4ef811e420 2026-02-25T15:01:02.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 10 monclient(hunting): my global_id is 4209 2026-02-25T15:01:02.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ec8003610 con 0x7f4ef811e420 2026-02-25T15:01:02.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1508987201 0 0) 0x7f4ef817a810 con 0x7f4ef811c500 2026-02-25T15:01:02.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 10 monclient(hunting): my global_id is 4219 2026-02-25T15:01:02.517 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ef8179630 con 0x7f4ef811c500 2026-02-25T15:01:02.517 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 652639504 0 0) 0x7f4ef817b9f0 con 0x7f4ef811ffc0 2026-02-25T15:01:02.517 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.517 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.517 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.517 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 10 monclient(hunting): my global_id is 4220 2026-02-25T15:01:02.517 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.500+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ef817a810 con 0x7f4ef811ffc0 2026-02-25T15:01:02.517 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.502+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3236520615 0 0) 0x7f4ec8003610 con 0x7f4ef811e420 2026-02-25T15:01:02.517 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.502+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4ef817b9f0 con 0x7f4ef811e420 2026-02-25T15:01:02.517 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.502+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4ee0002d00 con 0x7f4ef811e420 2026-02-25T15:01:02.517 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.502+0000 7f4eed7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:02.517 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.502+0000 7f4eed7fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:02.517 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.502+0000 7f4eed7fa640 10 monclient(hunting): dump: 2026-02-25T15:01:02.518 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:02.518 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.518 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.518 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.518 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.518 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:02.518 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.518 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.518 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.518 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.519 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.502+0000 7f4eed7fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:02.519 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.502+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2410566121 0 0) 0x7f4ef8179630 con 0x7f4ef811c500 2026-02-25T15:01:02.519 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.503+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4ec8003610 con 0x7f4ef811c500 2026-02-25T15:01:02.519 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.503+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4ee8003270 con 0x7f4ef811c500 2026-02-25T15:01:02.519 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.503+0000 7f4eed7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:02.519 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.503+0000 7f4eed7fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:02.519 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.503+0000 7f4eed7fa640 10 monclient(hunting): dump: 2026-02-25T15:01:02.519 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:02.520 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.520 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.520 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.520 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.520 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:02.520 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.520 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.520 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.520 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.520 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.503+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3473286558 0 0) 0x7f4ef817a810 con 0x7f4ef811ffc0 2026-02-25T15:01:02.520 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.503+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4ef8179630 con 0x7f4ef811ffc0 2026-02-25T15:01:02.520 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.503+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4ef4003430 con 0x7f4ef811ffc0 2026-02-25T15:01:02.520 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.503+0000 7f4eed7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:02.521 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.503+0000 7f4eed7fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:02.521 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.503+0000 7f4eed7fa640 10 monclient(hunting): dump: 2026-02-25T15:01:02.521 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:02.521 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.521 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.521 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.521 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.521 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:02.521 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.521 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.521 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.521 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.521 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 898218918 0 0) 0x7f4ef817b9f0 con 0x7f4ef811e420 2026-02-25T15:01:02.521 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 >> v1:10.20.193.94:6789/0 conn(0x7f4ef811ffc0 legacy=0x7f4ef8177f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.522 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 >> v1:10.20.193.93:6789/0 conn(0x7f4ef811c500 legacy=0x7f4ef812fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.522 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eed7fa640 10 monclient: _finish_hunting 0 2026-02-25T15:01:02.522 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eed7fa640 1 monclient: found mon.a 2026-02-25T15:01:02.522 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eed7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:02.522 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eed7fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:02.522 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef817cbd0 con 0x7f4ef811e420 2026-02-25T15:01:02.522 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eed7fa640 10 monclient: _finish_auth 0 2026-02-25T15:01:02.522 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eed7fa640 10 monclient: _check_auth_tickets 2026-02-25T15:01:02.522 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eed7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:02.522 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eff109640 5 monclient: authenticate success, global_id 4209 2026-02-25T15:01:02.522 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eff109640 10 monclient: _renew_subs 2026-02-25T15:01:02.522 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eff109640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:02.523 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eff109640 1 -- 10.20.193.189:0/257046860 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef817a9e0 con 0x7f4ef811e420 2026-02-25T15:01:02.523 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eff109640 10 client.4209.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:02.523 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eff109640 10 monclient: _renew_subs 2026-02-25T15:01:02.523 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eff109640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:02.523 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eff109640 1 -- 10.20.193.189:0/257046860 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4ef817afd0 con 0x7f4ef811e420 2026-02-25T15:01:02.523 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.504+0000 7f4eff109640 1 librados: init done 2026-02-25T15:01:02.523 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.505+0000 7f4eff109640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:02.523 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.505+0000 7f4eff109640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:02.523 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.505+0000 7f4eff109640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:02.523 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.505+0000 7f4eff109640 1 -- 10.20.193.189:0/257046860 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef818b560 con 0x7f4ef811e420 2026-02-25T15:01:02.594 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.594+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4ee0002fb0 con 0x7f4ef811e420 2026-02-25T15:01:02.595 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.594+0000 7f4eed7fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:02.595 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.594+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4ee0003bf0 con 0x7f4ef811e420 2026-02-25T15:01:02.595 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.594+0000 7f4eed7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:02.595 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.594+0000 7f4eed7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:02.595 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.594+0000 7f4eed7fa640 10 monclient: dump: 2026-02-25T15:01:02.595 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-25T15:01:02.595 INFO:teuthology.orchestra.run.trial189.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.595 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.595 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.596 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.596 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 3 2026-02-25T15:01:02.596 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.596 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.596 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.596 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:01:02.596 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.594+0000 7f4eeeffd640 4 set_mon_vals no callback set 2026-02-25T15:01:02.597 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.594+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 4) ==== 103168+0+0 (unknown 1070002493 0 0) 0x7f4ee001d0a0 con 0x7f4ef811e420 2026-02-25T15:01:02.597 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.595+0000 7f4efce7e640 10 monclient: get_auth_request con 0x7f4ec807a010 auth_method 0 2026-02-25T15:01:02.597 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.595+0000 7f4eed7fa640 10 client.4209.objecter ms_handle_connect 0x7f4ec807a010 2026-02-25T15:01:02.598 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.596+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(17..17 src has 1..17) ==== 2445+0+0 (unknown 1312276392 0 0) 0x7f4ee0059990 con 0x7f4ef811e420 2026-02-25T15:01:02.598 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.596+0000 7f4eed7fa640 10 client.4209.objecter ms_dispatch2 0x7f4ef8000b90 osd_map(17..17 src has 1..17) 2026-02-25T15:01:02.598 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.596+0000 7f4eed7fa640 3 client.4209.objecter handle_osd_map got epochs [17,17] > 0 2026-02-25T15:01:02.598 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.596+0000 7f4eed7fa640 3 client.4209.objecter handle_osd_map decoding full epoch 17 2026-02-25T15:01:02.598 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.596+0000 7f4eed7fa640 20 client.4209.objecter dump_active .. 0 homeless 2026-02-25T15:01:02.598 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.596+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f4ee005e100 con 0x7f4ef811e420 2026-02-25T15:01:02.598 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.596+0000 7f4eed7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:02.598 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.596+0000 7f4eed7fa640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:02.708 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.707+0000 7f4eff109640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "4445eda7-fb77-4f2c-8fda-77667df22b47", "id": 15}] 2026-02-25T15:01:02.709 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.707+0000 7f4eff109640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "4445eda7-fb77-4f2c-8fda-77667df22b47", "id": 15}] 2026-02-25T15:01:02.709 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.707+0000 7f4eff109640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:02.709 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.707+0000 7f4eff109640 1 -- 10.20.193.189:0/257046860 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd new", "uuid": "4445eda7-fb77-4f2c-8fda-77667df22b47", "id": 15} v 0) -- 0x7f4ef817b310 con 0x7f4ef811e420 2026-02-25T15:01:02.725 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eed7fa640 1 -- 10.20.193.189:0/257046860 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4445eda7-fb77-4f2c-8fda-77667df22b47", "id": 15}]=0 v18) ==== 113+0+3 (unknown 3012428783 0 1109645296) 0x7f4ee005da00 con 0x7f4ef811e420 2026-02-25T15:01:02.726 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eed7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "4445eda7-fb77-4f2c-8fda-77667df22b47", "id": 15}] 2026-02-25T15:01:02.726 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eed7fa640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:02.726 INFO:teuthology.orchestra.run.trial189.stdout:15 2026-02-25T15:01:02.727 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eff109640 10 librados: watch_flush enter 2026-02-25T15:01:02.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eff109640 10 librados: watch_flush exit 2026-02-25T15:01:02.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eff109640 20 client.4209.objecter shutdown clearing up homeless session... 2026-02-25T15:01:02.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eff109640 10 client.4209.objecter successfully canceled tick 2026-02-25T15:01:02.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eff109640 1 -- 10.20.193.189:0/257046860 >> v1:10.20.193.189:6800/3647375982 conn(0x7f4ec807a010 legacy=0x7f4ec809a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eff109640 10 monclient: shutdown 2026-02-25T15:01:02.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eff109640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:02.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eff109640 1 -- 10.20.193.189:0/257046860 >> v1:10.20.193.90:6789/0 conn(0x7f4ef811e420 legacy=0x7f4ef81767f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eff109640 1 -- 10.20.193.189:0/257046860 shutdown_connections 2026-02-25T15:01:02.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eff109640 1 -- 10.20.193.189:0/257046860 >> 10.20.193.189:0/257046860 conn(0x7f4ef80588d0 msgr2=0x7f4ef8058f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:02.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eff109640 1 -- 10.20.193.189:0/257046860 shutdown_connections 2026-02-25T15:01:02.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eff109640 1 -- 10.20.193.189:0/257046860 wait complete. 2026-02-25T15:01:02.729 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-25T15:01:02.725+0000 7f4eff109640 1 librados: shutdown 2026-02-25T15:01:02.734 DEBUG:tasks.ceph_manager:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.735 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-25T15:01:02.735 DEBUG:teuthology.orchestra.run.trial090:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 2026-02-25T15:01:02.737 INFO:tasks.ceph.osd.0:Started 2026-02-25T15:01:02.738 DEBUG:tasks.ceph_manager:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.738 INFO:tasks.ceph.osd.4:Restarting daemon 2026-02-25T15:01:02.738 DEBUG:teuthology.orchestra.run.trial090:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 2026-02-25T15:01:02.740 INFO:tasks.ceph.osd.4:Started 2026-02-25T15:01:02.740 DEBUG:tasks.ceph_manager:running osd.8 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.8.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.741 INFO:tasks.ceph.osd.8:Restarting daemon 2026-02-25T15:01:02.741 DEBUG:teuthology.orchestra.run.trial090:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.8.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 8 2026-02-25T15:01:02.743 INFO:tasks.ceph.osd.8:Started 2026-02-25T15:01:02.743 DEBUG:tasks.ceph_manager:running osd.12 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.12.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.743 INFO:tasks.ceph.osd.12:Restarting daemon 2026-02-25T15:01:02.744 DEBUG:teuthology.orchestra.run.trial090:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.12.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 12 2026-02-25T15:01:02.746 INFO:tasks.ceph.osd.12:Started 2026-02-25T15:01:02.746 DEBUG:tasks.ceph_manager:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.746 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-25T15:01:02.746 DEBUG:teuthology.orchestra.run.trial093:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 2026-02-25T15:01:02.749 INFO:tasks.ceph.osd.1:Started 2026-02-25T15:01:02.749 DEBUG:tasks.ceph_manager:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.749 INFO:tasks.ceph.osd.5:Restarting daemon 2026-02-25T15:01:02.749 DEBUG:teuthology.orchestra.run.trial093:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 2026-02-25T15:01:02.751 INFO:tasks.ceph.osd.5:Started 2026-02-25T15:01:02.751 DEBUG:tasks.ceph_manager:running osd.9 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.9.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.752 INFO:tasks.ceph.osd.9:Restarting daemon 2026-02-25T15:01:02.752 DEBUG:teuthology.orchestra.run.trial093:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.9.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 9 2026-02-25T15:01:02.754 INFO:tasks.ceph.osd.9:Started 2026-02-25T15:01:02.755 DEBUG:tasks.ceph_manager:running osd.13 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.13.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.755 INFO:tasks.ceph.osd.13:Restarting daemon 2026-02-25T15:01:02.755 DEBUG:teuthology.orchestra.run.trial093:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.13.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 13 2026-02-25T15:01:02.757 INFO:tasks.ceph.osd.13:Started 2026-02-25T15:01:02.757 DEBUG:tasks.ceph_manager:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.758 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-25T15:01:02.758 DEBUG:teuthology.orchestra.run.trial094:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 2026-02-25T15:01:02.760 INFO:tasks.ceph.osd.2:Started 2026-02-25T15:01:02.760 DEBUG:tasks.ceph_manager:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.760 INFO:tasks.ceph.osd.6:Restarting daemon 2026-02-25T15:01:02.761 DEBUG:teuthology.orchestra.run.trial094:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 2026-02-25T15:01:02.763 INFO:tasks.ceph.osd.6:Started 2026-02-25T15:01:02.763 DEBUG:tasks.ceph_manager:running osd.10 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.10.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.763 INFO:tasks.ceph.osd.10:Restarting daemon 2026-02-25T15:01:02.764 DEBUG:teuthology.orchestra.run.trial094:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.10.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 10 2026-02-25T15:01:02.766 INFO:tasks.ceph.osd.10:Started 2026-02-25T15:01:02.766 DEBUG:tasks.ceph_manager:running osd.14 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.14.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.766 INFO:tasks.ceph.osd.14:Restarting daemon 2026-02-25T15:01:02.766 DEBUG:teuthology.orchestra.run.trial094:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.14.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 14 2026-02-25T15:01:02.768 INFO:tasks.ceph.osd.14:Started 2026-02-25T15:01:02.768 DEBUG:tasks.ceph_manager:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.769 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-25T15:01:02.769 DEBUG:teuthology.orchestra.run.trial189:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 2026-02-25T15:01:02.771 INFO:tasks.ceph.osd.3:Started 2026-02-25T15:01:02.772 DEBUG:tasks.ceph_manager:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.772 INFO:tasks.ceph.osd.7:Restarting daemon 2026-02-25T15:01:02.772 DEBUG:teuthology.orchestra.run.trial189:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 2026-02-25T15:01:02.774 INFO:tasks.ceph.osd.7:Started 2026-02-25T15:01:02.774 DEBUG:tasks.ceph_manager:running osd.11 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.11.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.775 INFO:tasks.ceph.osd.11:Restarting daemon 2026-02-25T15:01:02.775 DEBUG:teuthology.orchestra.run.trial189:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.11.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 11 2026-02-25T15:01:02.777 INFO:tasks.ceph.osd.11:Started 2026-02-25T15:01:02.777 DEBUG:tasks.ceph_manager:running osd.15 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.15.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-25T15:01:02.778 INFO:tasks.ceph.osd.15:Restarting daemon 2026-02-25T15:01:02.778 DEBUG:teuthology.orchestra.run.trial189:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.15.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 15 2026-02-25T15:01:02.780 INFO:tasks.ceph.osd.15:Started 2026-02-25T15:01:02.781 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T15:01:02.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.852+0000 7f4597e82640 10 monclient: get_monmap_and_config 2026-02-25T15:01:02.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.852+0000 7f4597e82640 10 monclient: build_initial_monmap 2026-02-25T15:01:02.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.852+0000 7f4597e82640 10 monclient: monmap: 2026-02-25T15:01:02.855 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:02.855 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.855 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:02.854321+0000 2026-02-25T15:01:02.856 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:02.854321+0000 2026-02-25T15:01:02.856 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:02.856 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:02.856 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:02.856 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:02.856 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:02.856 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:02.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.853+0000 7f4597e82640 1 Processor -- start 2026-02-25T15:01:02.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.853+0000 7f4597e82640 1 -- start start 2026-02-25T15:01:02.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.853+0000 7f4597e82640 10 monclient: init 2026-02-25T15:01:02.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.853+0000 7f4597e82640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:02.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.853+0000 7f4597e82640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:02.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.853+0000 7f4597e82640 10 monclient(hunting): picked mon.noname-c con 0x7f4590058da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:02.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.853+0000 7f4597e82640 10 monclient(hunting): picked mon.noname-a con 0x7f4590059170 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:02.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.853+0000 7f4597e82640 10 monclient(hunting): picked mon.noname-b con 0x7f4590159ee0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:02.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.853+0000 7f4597e82640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f459005d6e0 con 0x7f4590059170 2026-02-25T15:01:02.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.853+0000 7f4597e82640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4590164100 con 0x7f4590159ee0 2026-02-25T15:01:02.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.853+0000 7f4597e82640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45900a8ff0 con 0x7f4590058da0 2026-02-25T15:01:02.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.853+0000 7f4597e82640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:02.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.853+0000 7f4597e82640 10 monclient(hunting): authenticate will time out at 2952.454699s 2026-02-25T15:01:02.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.854+0000 7f45963f8640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f4590159ee0 0x7f4590163340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:47470/0 (socket says 10.20.193.90:47470) 2026-02-25T15:01:02.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.854+0000 7f45963f8640 1 -- 10.20.193.90:0/722525943 learned_addr learned my addr 10.20.193.90:0/722525943 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:02.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.855+0000 7f4594bf5640 1 -- 10.20.193.90:0/722525943 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2244874597 0 0) 0x7f45900a8ff0 con 0x7f4590058da0 2026-02-25T15:01:02.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.856+0000 7f4594bf5640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.856+0000 7f4594bf5640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.856+0000 7f4594bf5640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.856+0000 7f4594bf5640 10 monclient(hunting): my global_id is 4223 2026-02-25T15:01:02.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.856+0000 7f4594bf5640 1 -- 10.20.193.90:0/722525943 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4578003610 con 0x7f4590058da0 2026-02-25T15:01:02.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4594bf5640 1 -- 10.20.193.90:0/722525943 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1766833511 0 0) 0x7f4578003610 con 0x7f4590058da0 2026-02-25T15:01:02.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4594bf5640 1 -- 10.20.193.90:0/722525943 >> v1:10.20.193.93:6789/0 conn(0x7f4590159ee0 legacy=0x7f4590163340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4594bf5640 1 -- 10.20.193.90:0/722525943 >> v1:10.20.193.90:6789/0 conn(0x7f4590059170 legacy=0x7f45901597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:02.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4594bf5640 10 monclient: _finish_hunting 0 2026-02-25T15:01:02.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4594bf5640 1 monclient: found mon.noname-c 2026-02-25T15:01:02.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4594bf5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:02.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4594bf5640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:02.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4594bf5640 1 -- 10.20.193.90:0/722525943 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45900a92d0 con 0x7f4590058da0 2026-02-25T15:01:02.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4594bf5640 10 monclient: _finish_auth 0 2026-02-25T15:01:02.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4594bf5640 10 monclient: _check_auth_tickets 2026-02-25T15:01:02.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4594bf5640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:02.862 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4594bf5640 1 -- 10.20.193.90:0/722525943 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4580002df0 con 0x7f4590058da0 2026-02-25T15:01:02.862 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4594bf5640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:02.862 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4594bf5640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:02.862 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4594bf5640 10 monclient: dump: 2026-02-25T15:01:02.862 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:02.862 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.862 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.862 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.862 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.862 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:02.862 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.863 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.863 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.863 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:02.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4597e82640 5 monclient: authenticate success, global_id 4223 2026-02-25T15:01:02.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.858+0000 7f4597e82640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:02.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4594bf5640 1 -- 10.20.193.90:0/722525943 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4580003000 con 0x7f4590058da0 2026-02-25T15:01:02.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4594bf5640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:02.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4594bf5640 1 -- 10.20.193.90:0/722525943 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4580003440 con 0x7f4590058da0 2026-02-25T15:01:02.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4594bf5640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:02.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4594bf5640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:02.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4594bf5640 10 monclient: dump: 2026-02-25T15:01:02.866 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:02.866 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.867 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.867 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.867 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.867 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:02.867 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.867 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.867 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.867 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:02.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4597e82640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:02.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4597e82640 4 set_mon_vals no callback set 2026-02-25T15:01:02.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4597e82640 10 monclient: shutdown 2026-02-25T15:01:02.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4597e82640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:02.868 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4597e82640 1 -- 10.20.193.90:0/722525943 >> v1:10.20.193.94:6789/0 conn(0x7f4590058da0 legacy=0x7f459005cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.868 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4597e82640 1 -- 10.20.193.90:0/722525943 shutdown_connections 2026-02-25T15:01:02.868 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4597e82640 1 -- 10.20.193.90:0/722525943 >> 10.20.193.90:0/722525943 conn(0x7f45900b65f0 msgr2=0x7f45900b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:02.868 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4597e82640 1 -- 10.20.193.90:0/722525943 shutdown_connections 2026-02-25T15:01:02.868 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4597e82640 1 -- 10.20.193.90:0/722525943 wait complete. 2026-02-25T15:01:02.868 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4597e82640 10 monclient: build_initial_monmap 2026-02-25T15:01:02.868 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.859+0000 7f4597e82640 10 monclient: monmap: 2026-02-25T15:01:02.868 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:02.868 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:02.868 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:02.868 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:02.868 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:02.869 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:02.869 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:02.869 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:02.869 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:02.869 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:02.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 1 librados: starting msgr at 2026-02-25T15:01:02.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 1 librados: starting objecter 2026-02-25T15:01:02.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:02.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 1 Processor -- start 2026-02-25T15:01:02.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 1 -- start start 2026-02-25T15:01:02.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 1 librados: setting wanted keys 2026-02-25T15:01:02.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 1 librados: calling monclient init 2026-02-25T15:01:02.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 10 monclient: init 2026-02-25T15:01:02.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:02.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:02.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 10 monclient(hunting): picked mon.noname-a con 0x7f4590058da0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:02.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 10 monclient(hunting): picked mon.noname-c con 0x7f4590059170 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:02.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 10 monclient(hunting): picked mon.noname-b con 0x7f4590159ee0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:02.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45901305f0 con 0x7f4590058da0 2026-02-25T15:01:02.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45901308d0 con 0x7f4590159ee0 2026-02-25T15:01:02.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4590130bb0 con 0x7f4590059170 2026-02-25T15:01:02.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:02.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.860+0000 7f4597e82640 10 monclient(hunting): authenticate will time out at 2952.461599s 2026-02-25T15:01:02.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.861+0000 7f45963f8640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f4590159ee0 0x7f4590120af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:47486/0 (socket says 10.20.193.90:47486) 2026-02-25T15:01:02.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.861+0000 7f45963f8640 1 -- 10.20.193.90:0/2755350530 learned_addr learned my addr 10.20.193.90:0/2755350530 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:02.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.861+0000 7f457e7fc640 10 client.?.objecter ms_handle_connect 0x7f4590159ee0 2026-02-25T15:01:02.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.861+0000 7f457e7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:02.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.861+0000 7f457e7fc640 10 client.?.objecter ms_handle_connect 0x7f4590059170 2026-02-25T15:01:02.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.861+0000 7f457e7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:02.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.862+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 403606612 0 0) 0x7f4590130bb0 con 0x7f4590059170 2026-02-25T15:01:02.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.862+0000 7f457e7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.862+0000 7f457e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.862+0000 7f457e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.862+0000 7f457e7fc640 10 monclient(hunting): my global_id is 4226 2026-02-25T15:01:02.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.862+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4560003610 con 0x7f4590059170 2026-02-25T15:01:02.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.864+0000 7f457e7fc640 10 client.?.objecter ms_handle_connect 0x7f4590058da0 2026-02-25T15:01:02.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.864+0000 7f457e7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:02.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.864+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399040814 0 0) 0x7f45901308d0 con 0x7f4590159ee0 2026-02-25T15:01:02.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.864+0000 7f457e7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.864+0000 7f457e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.864+0000 7f457e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.864+0000 7f457e7fc640 10 monclient(hunting): my global_id is 4222 2026-02-25T15:01:02.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.864+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4590130bb0 con 0x7f4590159ee0 2026-02-25T15:01:02.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.865+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2989494568 0 0) 0x7f4560003610 con 0x7f4590059170 2026-02-25T15:01:02.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.865+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f45901308d0 con 0x7f4590059170 2026-02-25T15:01:02.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.865+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4584002f70 con 0x7f4590059170 2026-02-25T15:01:02.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.865+0000 7f457e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:02.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.865+0000 7f457e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:02.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.865+0000 7f457e7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:02.875 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:02.875 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.875 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.875 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.875 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.875 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:02.875 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.875 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.875 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.876 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:02.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.865+0000 7f457e7fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:02.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.866+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 151684455 0 0) 0x7f45901305f0 con 0x7f4590058da0 2026-02-25T15:01:02.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.866+0000 7f457e7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:02.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.866+0000 7f457e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:02.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.866+0000 7f457e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:02.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.866+0000 7f457e7fc640 10 monclient(hunting): my global_id is 4212 2026-02-25T15:01:02.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.866+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4560003610 con 0x7f4590058da0 2026-02-25T15:01:02.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1477201458 0 0) 0x7f45901308d0 con 0x7f4590059170 2026-02-25T15:01:02.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 >> v1:10.20.193.93:6789/0 conn(0x7f4590159ee0 legacy=0x7f4590120af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 >> v1:10.20.193.90:6789/0 conn(0x7f4590058da0 legacy=0x7f4590131f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:02.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f457e7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:01:02.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f457e7fc640 1 monclient: found mon.c 2026-02-25T15:01:02.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f457e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:02.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f457e7fc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:02.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4590130e90 con 0x7f4590059170 2026-02-25T15:01:02.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f457e7fc640 10 monclient: _finish_auth 0 2026-02-25T15:01:02.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f457e7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:01:02.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f457e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:02.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f4597e82640 5 monclient: authenticate success, global_id 4226 2026-02-25T15:01:02.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f4597e82640 10 monclient: _renew_subs 2026-02-25T15:01:02.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f4597e82640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:02.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f4597e82640 1 -- 10.20.193.90:0/2755350530 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4590131170 con 0x7f4590059170 2026-02-25T15:01:02.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f4597e82640 10 client.4226.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:02.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f4597e82640 10 monclient: _renew_subs 2026-02-25T15:01:02.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f4597e82640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:02.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f4597e82640 1 -- 10.20.193.90:0/2755350530 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f45901326a0 con 0x7f4590059170 2026-02-25T15:01:02.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f4597e82640 1 librados: init done 2026-02-25T15:01:02.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f4597e82640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:02.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f4597e82640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:02.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f4597e82640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:02.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.867+0000 7f4597e82640 1 -- 10.20.193.90:0/2755350530 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45901545b0 con 0x7f4590059170 2026-02-25T15:01:02.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.873+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f45840035f0 con 0x7f4590059170 2026-02-25T15:01:02.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.873+0000 7f457e7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:02.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.873+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4584003940 con 0x7f4590059170 2026-02-25T15:01:02.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.873+0000 7f457e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:02.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.873+0000 7f457e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:02.880 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.873+0000 7f457e7fc640 10 monclient: dump: 2026-02-25T15:01:02.880 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:02.880 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:02.880 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.880 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:02.880 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:02.880 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:02.880 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:02.880 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:02.880 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:02.880 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:02.880 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.873+0000 7f457ffff640 4 set_mon_vals no callback set 2026-02-25T15:01:02.880 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.876+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 4) ==== 103168+0+0 (unknown 1070002493 0 0) 0x7f458401cdf0 con 0x7f4590059170 2026-02-25T15:01:02.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.877+0000 7f4595bf7640 10 monclient: get_auth_request con 0x7f4560079ed0 auth_method 0 2026-02-25T15:01:02.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.877+0000 7f457e7fc640 10 client.4226.objecter ms_handle_connect 0x7f4560079ed0 2026-02-25T15:01:02.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.878+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(18..18 src has 1..18) ==== 2560+0+0 (unknown 1923581746 0 0) 0x7f4584059c10 con 0x7f4590059170 2026-02-25T15:01:02.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.878+0000 7f457e7fc640 10 client.4226.objecter ms_dispatch2 0x7f4590000b90 osd_map(18..18 src has 1..18) 2026-02-25T15:01:02.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.878+0000 7f457e7fc640 3 client.4226.objecter handle_osd_map got epochs [18,18] > 0 2026-02-25T15:01:02.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.878+0000 7f457e7fc640 3 client.4226.objecter handle_osd_map decoding full epoch 18 2026-02-25T15:01:02.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.878+0000 7f457e7fc640 20 client.4226.objecter dump_active .. 0 homeless 2026-02-25T15:01:02.966 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.964+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f458405e380 con 0x7f4590059170 2026-02-25T15:01:02.966 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.964+0000 7f457e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:02.966 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:02.964+0000 7f457e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:03.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.072+0000 7f4597e82640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:03.075 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.072+0000 7f4597e82640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:03.075 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.072+0000 7f4597e82640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:03.075 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.072+0000 7f4597e82640 1 -- 10.20.193.90:0/2755350530 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4590157570 con 0x7f4590059170 2026-02-25T15:01:03.096 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.094+0000 7f457e7fc640 1 -- 10.20.193.90:0/2755350530 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12877 (unknown 3338201251 0 373084174) 0x7f458405dc80 con 0x7f4590059170 2026-02-25T15:01:03.096 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.094+0000 7f457e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:03.096 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.094+0000 7f457e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:03.096 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:01:03.096 INFO:teuthology.orchestra.run.trial090.stdout:{"epoch":18,"fsid":"63c679b5-d219-4797-8adb-7340f3fd6819","created":"2026-02-25T15:00:56.256818+0000","modified":"2026-02-25T15:01:02.711189+0000","last_up_change":"0.000000","last_in_change":"2026-02-25T15:01:02.711189+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ad97006e-f5c3-4e36-97a0-7fa2b250344d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"25b02f30-96b9-43a3-9691-f06a1292e36a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aeed908b-d483-498b-83ba-632e51dd05e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c7a30346-685f-4bd2-acaa-00b98ad3284c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d6c97301-55cc-4579-bc92-cb240c5ebe24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e3117058-2f0c-466b-a302-061fc4c120fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f4b4a3ce-0a0b-4386-a433-dcbf65558ecf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3337e549-6b6e-444e-997b-18a19beb5640","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"b292f7c4-9a41-4e48-876c-29281c5ec73b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"954a7cbb-2f06-4cb9-ae68-380ac58a9f21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"95ed1c74-8ca5-4317-9ef7-fc657b3cda70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"8f617b59-e8d1-42fb-8787-64e8c2117954","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"4bad4371-2ca7-4023-8406-0275011eda9c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"5d1ca952-93a5-43f3-9780-8bc85f9ed3d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"50853d20-2756-4720-b7aa-3020ded29189","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"4445eda7-fb77-4f2c-8fda-77667df22b47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T15:01:03.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.096+0000 7f4597e82640 10 librados: watch_flush enter 2026-02-25T15:01:03.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.096+0000 7f4597e82640 10 librados: watch_flush exit 2026-02-25T15:01:03.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.096+0000 7f4597e82640 20 client.4226.objecter shutdown clearing up homeless session... 2026-02-25T15:01:03.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.096+0000 7f4597e82640 10 client.4226.objecter successfully canceled tick 2026-02-25T15:01:03.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.096+0000 7f4597e82640 1 -- 10.20.193.90:0/2755350530 >> v1:10.20.193.189:6800/3647375982 conn(0x7f4560079ed0 legacy=0x7f456009a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:03.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.096+0000 7f4597e82640 10 monclient: shutdown 2026-02-25T15:01:03.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.096+0000 7f4597e82640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:03.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.096+0000 7f4597e82640 1 -- 10.20.193.90:0/2755350530 >> v1:10.20.193.94:6789/0 conn(0x7f4590059170 legacy=0x7f45900b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:03.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.096+0000 7f4597e82640 1 -- 10.20.193.90:0/2755350530 shutdown_connections 2026-02-25T15:01:03.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.096+0000 7f4597e82640 1 -- 10.20.193.90:0/2755350530 >> 10.20.193.90:0/2755350530 conn(0x7f45900b65f0 msgr2=0x7f45900a7840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:03.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.096+0000 7f4597e82640 1 -- 10.20.193.90:0/2755350530 shutdown_connections 2026-02-25T15:01:03.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.096+0000 7f4597e82640 1 -- 10.20.193.90:0/2755350530 wait complete. 2026-02-25T15:01:03.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:03.096+0000 7f4597e82640 1 librados: shutdown 2026-02-25T15:01:03.108 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-25T15:01:03.108 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-25T15:01:04.722 DEBUG:teuthology.orchestra.run.trial090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-25T15:01:04.781 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.779+0000 7fe3a80a4640 10 monclient: get_monmap_and_config 2026-02-25T15:01:04.782 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.779+0000 7fe3a80a4640 10 monclient: build_initial_monmap 2026-02-25T15:01:04.782 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.779+0000 7fe3a80a4640 10 monclient: monmap: 2026-02-25T15:01:04.782 INFO:teuthology.misc.health.trial090.stderr:epoch 0 2026-02-25T15:01:04.782 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:04.782 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:01:04.781388+0000 2026-02-25T15:01:04.782 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:01:04.781388+0000 2026-02-25T15:01:04.782 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:04.782 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:04.783 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:04.783 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:04.783 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:04.783 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:04.784 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.780+0000 7fe3a80a4640 1 Processor -- start 2026-02-25T15:01:04.784 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.780+0000 7fe3a80a4640 1 -- start start 2026-02-25T15:01:04.785 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.780+0000 7fe3a80a4640 10 monclient: init 2026-02-25T15:01:04.785 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.780+0000 7fe3a80a4640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:04.785 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.780+0000 7fe3a80a4640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:04.785 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.780+0000 7fe3a80a4640 10 monclient(hunting): picked mon.noname-b con 0x7fe3a0058da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:04.785 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.780+0000 7fe3a80a4640 10 monclient(hunting): picked mon.noname-c con 0x7fe3a00b0410 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:04.785 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.780+0000 7fe3a80a4640 10 monclient(hunting): picked mon.noname-a con 0x7fe3a01626a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:04.785 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.780+0000 7fe3a80a4640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a01655c0 con 0x7fe3a01626a0 2026-02-25T15:01:04.785 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.780+0000 7fe3a80a4640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a00b12b0 con 0x7fe3a0058da0 2026-02-25T15:01:04.785 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.780+0000 7fe3a80a4640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a00b1590 con 0x7fe3a00b0410 2026-02-25T15:01:04.785 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.780+0000 7fe3a80a4640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:04.785 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.780+0000 7fe3a80a4640 10 monclient(hunting): authenticate will time out at 2954.381894s 2026-02-25T15:01:04.785 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.781+0000 7fe3a5e19640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fe3a0058da0 0x7fe3a01647a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:48870/0 (socket says 10.20.193.90:48870) 2026-02-25T15:01:04.786 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.781+0000 7fe3a5e19640 1 -- 10.20.193.90:0/1547133042 learned_addr learned my addr 10.20.193.90:0/1547133042 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:04.790 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 1 -- 10.20.193.90:0/1547133042 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 482393935 0 0) 0x7fe3a00b12b0 con 0x7fe3a0058da0 2026-02-25T15:01:04.790 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 10 monclient(hunting): _negotiate 2026-02-25T15:01:04.790 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:04.790 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:04.790 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 10 monclient(hunting): my global_id is 4225 2026-02-25T15:01:04.790 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 1 -- 10.20.193.90:0/1547133042 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe388003610 con 0x7fe3a0058da0 2026-02-25T15:01:04.790 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 1 -- 10.20.193.90:0/1547133042 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3426511553 0 0) 0x7fe3a01655c0 con 0x7fe3a01626a0 2026-02-25T15:01:04.790 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 10 monclient(hunting): _negotiate 2026-02-25T15:01:04.790 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:04.790 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:04.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 10 monclient(hunting): my global_id is 4215 2026-02-25T15:01:04.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 1 -- 10.20.193.90:0/1547133042 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3a00b12b0 con 0x7fe3a01626a0 2026-02-25T15:01:04.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 1 -- 10.20.193.90:0/1547133042 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3867937708 0 0) 0x7fe3a00b1590 con 0x7fe3a00b0410 2026-02-25T15:01:04.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 10 monclient(hunting): _negotiate 2026-02-25T15:01:04.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:04.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:04.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 10 monclient(hunting): my global_id is 4229 2026-02-25T15:01:04.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.783+0000 7fe3a4e17640 1 -- 10.20.193.90:0/1547133042 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3a01655c0 con 0x7fe3a00b0410 2026-02-25T15:01:04.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a4e17640 1 -- 10.20.193.90:0/1547133042 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 598537804 0 0) 0x7fe388003610 con 0x7fe3a0058da0 2026-02-25T15:01:04.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a4e17640 1 -- 10.20.193.90:0/1547133042 >> v1:10.20.193.94:6789/0 conn(0x7fe3a00b0410 legacy=0x7fe3a0164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:04.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a4e17640 1 -- 10.20.193.90:0/1547133042 >> v1:10.20.193.90:6789/0 conn(0x7fe3a01626a0 legacy=0x7fe3a01c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:04.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a4e17640 10 monclient: _finish_hunting 0 2026-02-25T15:01:04.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a4e17640 1 monclient: found mon.noname-b 2026-02-25T15:01:04.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a4e17640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:04.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a4e17640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:04.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a4e17640 1 -- 10.20.193.90:0/1547133042 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a00b1870 con 0x7fe3a0058da0 2026-02-25T15:01:04.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a4e17640 10 monclient: _finish_auth 0 2026-02-25T15:01:04.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a4e17640 10 monclient: _check_auth_tickets 2026-02-25T15:01:04.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a4e17640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:04.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a4e17640 1 -- 10.20.193.90:0/1547133042 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe390002df0 con 0x7fe3a0058da0 2026-02-25T15:01:04.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a4e17640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:04.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a4e17640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:04.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a4e17640 10 monclient: dump: 2026-02-25T15:01:04.792 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:04.792 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:04.793 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:04.793 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:04.793 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:04.793 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:04.793 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:04.793 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:04.793 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:04.793 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:04.793 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a80a4640 5 monclient: authenticate success, global_id 4225 2026-02-25T15:01:04.793 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.784+0000 7fe3a80a4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:04.793 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.785+0000 7fe3a4e17640 1 -- 10.20.193.90:0/1547133042 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe390002fc0 con 0x7fe3a0058da0 2026-02-25T15:01:04.793 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.785+0000 7fe3a4e17640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:04.794 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.785+0000 7fe3a4e17640 1 -- 10.20.193.90:0/1547133042 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe390003400 con 0x7fe3a0058da0 2026-02-25T15:01:04.794 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.785+0000 7fe3a4e17640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:04.794 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.785+0000 7fe3a4e17640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:04.794 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.785+0000 7fe3a4e17640 10 monclient: dump: 2026-02-25T15:01:04.794 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:04.794 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:04.794 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:04.794 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:04.794 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:04.794 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:04.794 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:04.794 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:04.794 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:04.795 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:04.795 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.785+0000 7fe3a80a4640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:04.795 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.785+0000 7fe3a80a4640 4 set_mon_vals no callback set 2026-02-25T15:01:04.795 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.785+0000 7fe3a80a4640 10 monclient: shutdown 2026-02-25T15:01:04.795 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.785+0000 7fe3a80a4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:04.795 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.785+0000 7fe3a80a4640 1 -- 10.20.193.90:0/1547133042 >> v1:10.20.193.93:6789/0 conn(0x7fe3a0058da0 legacy=0x7fe3a01647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:04.795 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.786+0000 7fe3a80a4640 1 -- 10.20.193.90:0/1547133042 shutdown_connections 2026-02-25T15:01:04.795 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.786+0000 7fe3a80a4640 1 -- 10.20.193.90:0/1547133042 >> 10.20.193.90:0/1547133042 conn(0x7fe3a00b65f0 msgr2=0x7fe3a00b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:04.795 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.786+0000 7fe3a80a4640 1 -- 10.20.193.90:0/1547133042 shutdown_connections 2026-02-25T15:01:04.795 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.786+0000 7fe3a80a4640 1 -- 10.20.193.90:0/1547133042 wait complete. 2026-02-25T15:01:04.795 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.786+0000 7fe3a80a4640 10 monclient: build_initial_monmap 2026-02-25T15:01:04.795 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.786+0000 7fe3a80a4640 10 monclient: monmap: 2026-02-25T15:01:04.796 INFO:teuthology.misc.health.trial090.stderr:epoch 0 2026-02-25T15:01:04.796 INFO:teuthology.misc.health.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:04.796 INFO:teuthology.misc.health.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:04.796 INFO:teuthology.misc.health.trial090.stderr:created 0.000000 2026-02-25T15:01:04.796 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:04.796 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 1 2026-02-25T15:01:04.796 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:04.796 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:04.796 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:04.796 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:04.796 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.786+0000 7fe3a80a4640 1 librados: starting msgr at 2026-02-25T15:01:04.796 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.786+0000 7fe3a80a4640 1 librados: starting objecter 2026-02-25T15:01:04.796 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.786+0000 7fe3a80a4640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:04.797 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.786+0000 7fe3a80a4640 1 Processor -- start 2026-02-25T15:01:04.797 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.786+0000 7fe3a80a4640 1 -- start start 2026-02-25T15:01:04.797 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.786+0000 7fe3a80a4640 1 librados: setting wanted keys 2026-02-25T15:01:04.797 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.786+0000 7fe3a80a4640 1 librados: calling monclient init 2026-02-25T15:01:04.797 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.786+0000 7fe3a80a4640 10 monclient: init 2026-02-25T15:01:04.797 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.787+0000 7fe3a80a4640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:04.797 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.787+0000 7fe3a80a4640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:04.797 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.787+0000 7fe3a80a4640 10 monclient(hunting): picked mon.noname-b con 0x7fe3a0058da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:04.797 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.787+0000 7fe3a80a4640 10 monclient(hunting): picked mon.noname-a con 0x7fe3a00b0410 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:04.797 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.787+0000 7fe3a80a4640 10 monclient(hunting): picked mon.noname-c con 0x7fe3a01626a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:04.797 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.787+0000 7fe3a80a4640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a013b580 con 0x7fe3a00b0410 2026-02-25T15:01:04.797 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.787+0000 7fe3a80a4640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a013ec90 con 0x7fe3a0058da0 2026-02-25T15:01:04.797 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.787+0000 7fe3a80a4640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a013ef70 con 0x7fe3a01626a0 2026-02-25T15:01:04.798 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.787+0000 7fe3a80a4640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:04.798 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.787+0000 7fe3a80a4640 10 monclient(hunting): authenticate will time out at 2954.388456s 2026-02-25T15:01:04.798 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.787+0000 7fe3a5e19640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fe3a0058da0 0x7fe3a01fa240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:48872/0 (socket says 10.20.193.90:48872) 2026-02-25T15:01:04.798 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.787+0000 7fe3a5e19640 1 -- 10.20.193.90:0/4082291823 learned_addr learned my addr 10.20.193.90:0/4082291823 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:04.798 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.788+0000 7fe38e7fc640 10 client.?.objecter ms_handle_connect 0x7fe3a0058da0 2026-02-25T15:01:04.798 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.788+0000 7fe38e7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:04.803 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.788+0000 7fe38e7fc640 10 client.?.objecter ms_handle_connect 0x7fe3a00b0410 2026-02-25T15:01:04.803 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.788+0000 7fe38e7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:04.803 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.788+0000 7fe38e7fc640 10 client.?.objecter ms_handle_connect 0x7fe3a01626a0 2026-02-25T15:01:04.803 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.788+0000 7fe38e7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:04.803 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2773274000 0 0) 0x7fe3a013ec90 con 0x7fe3a0058da0 2026-02-25T15:01:04.803 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:04.803 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:04.803 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:04.804 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 10 monclient(hunting): my global_id is 4228 2026-02-25T15:01:04.804 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe370003610 con 0x7fe3a0058da0 2026-02-25T15:01:04.804 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3323219282 0 0) 0x7fe3a01655c0 con 0x7fe3a00b0410 2026-02-25T15:01:04.804 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:04.804 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:04.804 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:04.804 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 10 monclient(hunting): my global_id is 4218 2026-02-25T15:01:04.804 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3a013ec90 con 0x7fe3a00b0410 2026-02-25T15:01:04.804 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2669494219 0 0) 0x7fe3a013ef70 con 0x7fe3a01626a0 2026-02-25T15:01:04.804 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:04.804 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:04.804 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:04.805 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 10 monclient(hunting): my global_id is 4232 2026-02-25T15:01:04.805 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.789+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3a01655c0 con 0x7fe3a01626a0 2026-02-25T15:01:04.805 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.791+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3083251179 0 0) 0x7fe370003610 con 0x7fe3a0058da0 2026-02-25T15:01:04.805 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.791+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe3a013ef70 con 0x7fe3a0058da0 2026-02-25T15:01:04.805 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.791+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe390003310 con 0x7fe3a0058da0 2026-02-25T15:01:04.805 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.791+0000 7fe38e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:04.805 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.791+0000 7fe38e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:04.805 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.791+0000 7fe38e7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:04.805 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:04.805 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:04.805 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:04.805 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:04.806 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:04.806 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:04.806 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:04.806 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:04.806 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:04.806 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:04.806 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.791+0000 7fe38e7fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:04.806 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.792+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 51581802 0 0) 0x7fe3a01655c0 con 0x7fe3a01626a0 2026-02-25T15:01:04.806 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.792+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe370003610 con 0x7fe3a01626a0 2026-02-25T15:01:04.806 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.792+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe39c003370 con 0x7fe3a01626a0 2026-02-25T15:01:04.806 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.792+0000 7fe38e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:04.806 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.792+0000 7fe38e7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:04.807 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.792+0000 7fe38e7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:04.807 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:04.807 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:04.807 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:04.807 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:04.807 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:04.807 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:04.807 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:04.807 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:04.807 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:04.807 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:04.807 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.792+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1589380964 0 0) 0x7fe3a013ec90 con 0x7fe3a00b0410 2026-02-25T15:01:04.808 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.792+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe3a01655c0 con 0x7fe3a00b0410 2026-02-25T15:01:04.808 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.792+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe394003060 con 0x7fe3a00b0410 2026-02-25T15:01:04.808 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.792+0000 7fe38e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:04.808 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.792+0000 7fe38e7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:04.808 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.792+0000 7fe38e7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:04.808 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:04.808 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:04.808 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:04.808 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:04.808 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:04.808 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:04.808 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:04.808 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:04.809 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:04.809 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:04.809 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1601113938 0 0) 0x7fe3a013ef70 con 0x7fe3a0058da0 2026-02-25T15:01:04.809 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 >> v1:10.20.193.94:6789/0 conn(0x7fe3a01626a0 legacy=0x7fe3a01fc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:04.809 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 >> v1:10.20.193.90:6789/0 conn(0x7fe3a00b0410 legacy=0x7fe3a0139190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:04.809 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe38e7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:01:04.809 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe38e7fc640 1 monclient: found mon.b 2026-02-25T15:01:04.809 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe38e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:04.809 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe38e7fc640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:04.809 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a013f250 con 0x7fe3a0058da0 2026-02-25T15:01:04.809 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe38e7fc640 10 monclient: _finish_auth 0 2026-02-25T15:01:04.809 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe38e7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:01:04.809 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe38e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:04.810 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe3a80a4640 5 monclient: authenticate success, global_id 4228 2026-02-25T15:01:04.810 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe3a80a4640 10 monclient: _renew_subs 2026-02-25T15:01:04.810 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe3a80a4640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:04.810 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe3a80a4640 1 -- 10.20.193.90:0/4082291823 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a013f530 con 0x7fe3a0058da0 2026-02-25T15:01:04.810 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe3a80a4640 10 client.4228.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:04.810 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe3a80a4640 10 monclient: _renew_subs 2026-02-25T15:01:04.810 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe3a80a4640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:04.810 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe3a80a4640 1 -- 10.20.193.90:0/4082291823 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe3a0140010 con 0x7fe3a0058da0 2026-02-25T15:01:04.810 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe3a80a4640 1 librados: init done 2026-02-25T15:01:04.810 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe3a80a4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:04.810 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe3a80a4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:04.810 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe3a80a4640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:04.811 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.793+0000 7fe3a80a4640 1 -- 10.20.193.90:0/4082291823 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a0164050 con 0x7fe3a0058da0 2026-02-25T15:01:04.811 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.800+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe3900029a0 con 0x7fe3a0058da0 2026-02-25T15:01:04.811 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.800+0000 7fe38e7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:04.811 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.800+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe390004b80 con 0x7fe3a0058da0 2026-02-25T15:01:04.811 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.800+0000 7fe38e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:04.811 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.800+0000 7fe38e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:04.811 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.800+0000 7fe38ffff640 4 set_mon_vals no callback set 2026-02-25T15:01:04.811 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.800+0000 7fe38e7fc640 10 monclient: dump: 2026-02-25T15:01:04.811 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:04.811 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:04.811 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:04.811 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:04.812 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:04.812 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:04.812 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:04.812 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:04.812 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:04.812 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:04.813 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.804+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 4) ==== 103168+0+0 (unknown 1070002493 0 0) 0x7fe39001de90 con 0x7fe3a0058da0 2026-02-25T15:01:04.813 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.804+0000 7fe3a5618640 10 monclient: get_auth_request con 0x7fe37007a100 auth_method 0 2026-02-25T15:01:04.813 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.805+0000 7fe38e7fc640 10 client.4228.objecter ms_handle_connect 0x7fe37007a100 2026-02-25T15:01:04.813 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.806+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(18..18 src has 1..18) ==== 2560+0+0 (unknown 1923581746 0 0) 0x7fe39005ac10 con 0x7fe3a0058da0 2026-02-25T15:01:04.813 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.806+0000 7fe38e7fc640 10 client.4228.objecter ms_dispatch2 0x7fe3a0000b90 osd_map(18..18 src has 1..18) 2026-02-25T15:01:04.813 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.806+0000 7fe38e7fc640 3 client.4228.objecter handle_osd_map got epochs [18,18] > 0 2026-02-25T15:01:04.813 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.806+0000 7fe38e7fc640 3 client.4228.objecter handle_osd_map decoding full epoch 18 2026-02-25T15:01:04.814 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.806+0000 7fe38e7fc640 20 client.4228.objecter dump_active .. 0 homeless 2026-02-25T15:01:04.894 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.892+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fe39005f380 con 0x7fe3a0058da0 2026-02-25T15:01:04.894 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.892+0000 7fe38e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:04.894 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:04.892+0000 7fe38e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:05.005 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.003+0000 7fe3a80a4640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:05.005 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.003+0000 7fe3a80a4640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:05.005 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.003+0000 7fe3a80a4640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:05.005 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.003+0000 7fe3a80a4640 1 -- 10.20.193.90:0/4082291823 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3a00b6ca0 con 0x7fe3a0058da0 2026-02-25T15:01:05.027 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.025+0000 7fe38e7fc640 1 -- 10.20.193.90:0/4082291823 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12877 (unknown 3338201251 0 373084174) 0x7fe39005ec80 con 0x7fe3a0058da0 2026-02-25T15:01:05.027 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.025+0000 7fe38e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:05.028 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.025+0000 7fe38e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:05.028 INFO:teuthology.misc.health.trial090.stdout: 2026-02-25T15:01:05.028 INFO:teuthology.misc.health.trial090.stdout:{"epoch":18,"fsid":"63c679b5-d219-4797-8adb-7340f3fd6819","created":"2026-02-25T15:00:56.256818+0000","modified":"2026-02-25T15:01:02.711189+0000","last_up_change":"0.000000","last_in_change":"2026-02-25T15:01:02.711189+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ad97006e-f5c3-4e36-97a0-7fa2b250344d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"25b02f30-96b9-43a3-9691-f06a1292e36a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aeed908b-d483-498b-83ba-632e51dd05e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c7a30346-685f-4bd2-acaa-00b98ad3284c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d6c97301-55cc-4579-bc92-cb240c5ebe24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e3117058-2f0c-466b-a302-061fc4c120fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f4b4a3ce-0a0b-4386-a433-dcbf65558ecf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3337e549-6b6e-444e-997b-18a19beb5640","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"b292f7c4-9a41-4e48-876c-29281c5ec73b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"954a7cbb-2f06-4cb9-ae68-380ac58a9f21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"95ed1c74-8ca5-4317-9ef7-fc657b3cda70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"8f617b59-e8d1-42fb-8787-64e8c2117954","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"4bad4371-2ca7-4023-8406-0275011eda9c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"5d1ca952-93a5-43f3-9780-8bc85f9ed3d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"50853d20-2756-4720-b7aa-3020ded29189","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"4445eda7-fb77-4f2c-8fda-77667df22b47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T15:01:05.031 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.028+0000 7fe3a80a4640 10 librados: watch_flush enter 2026-02-25T15:01:05.031 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.028+0000 7fe3a80a4640 10 librados: watch_flush exit 2026-02-25T15:01:05.031 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.028+0000 7fe3a80a4640 20 client.4228.objecter shutdown clearing up homeless session... 2026-02-25T15:01:05.031 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.028+0000 7fe3a80a4640 10 client.4228.objecter successfully canceled tick 2026-02-25T15:01:05.031 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.028+0000 7fe3a80a4640 1 -- 10.20.193.90:0/4082291823 >> v1:10.20.193.189:6800/3647375982 conn(0x7fe37007a100 legacy=0x7fe37009a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:05.031 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.028+0000 7fe3a80a4640 10 monclient: shutdown 2026-02-25T15:01:05.032 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.028+0000 7fe3a80a4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:05.032 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.028+0000 7fe3a80a4640 1 -- 10.20.193.90:0/4082291823 >> v1:10.20.193.93:6789/0 conn(0x7fe3a0058da0 legacy=0x7fe3a01fa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:05.032 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.028+0000 7fe3a80a4640 1 -- 10.20.193.90:0/4082291823 shutdown_connections 2026-02-25T15:01:05.032 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.028+0000 7fe3a80a4640 1 -- 10.20.193.90:0/4082291823 >> 10.20.193.90:0/4082291823 conn(0x7fe3a00b65f0 msgr2=0x7fe3a0059bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:05.032 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.028+0000 7fe3a80a4640 1 -- 10.20.193.90:0/4082291823 shutdown_connections 2026-02-25T15:01:05.032 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.028+0000 7fe3a80a4640 1 -- 10.20.193.90:0/4082291823 wait complete. 2026-02-25T15:01:05.032 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:05.028+0000 7fe3a80a4640 1 librados: shutdown 2026-02-25T15:01:05.040 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-25T15:01:12.653 DEBUG:teuthology.orchestra.run.trial090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-25T15:01:12.716 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.715+0000 7f4d90f67640 10 monclient: get_monmap_and_config 2026-02-25T15:01:12.716 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.715+0000 7f4d90f67640 10 monclient: build_initial_monmap 2026-02-25T15:01:12.716 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.715+0000 7f4d90f67640 10 monclient: monmap: 2026-02-25T15:01:12.716 INFO:teuthology.misc.health.trial090.stderr:epoch 0 2026-02-25T15:01:12.716 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:12.716 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:01:12.715742+0000 2026-02-25T15:01:12.717 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:01:12.715742+0000 2026-02-25T15:01:12.717 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:12.717 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:12.717 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:12.717 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:12.717 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:12.717 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:12.719 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.715+0000 7f4d90f67640 1 Processor -- start 2026-02-25T15:01:12.719 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.715+0000 7f4d90f67640 1 -- start start 2026-02-25T15:01:12.719 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.715+0000 7f4d90f67640 10 monclient: init 2026-02-25T15:01:12.719 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.716+0000 7f4d90f67640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:12.719 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.716+0000 7f4d90f67640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:12.720 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.716+0000 7f4d90f67640 10 monclient(hunting): picked mon.noname-b con 0x7f4d8c058da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:12.720 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.716+0000 7f4d90f67640 10 monclient(hunting): picked mon.noname-a con 0x7f4d8c0b0410 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:12.720 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.716+0000 7f4d90f67640 10 monclient(hunting): picked mon.noname-c con 0x7f4d8c1626a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:12.720 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.716+0000 7f4d90f67640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d8c1655c0 con 0x7f4d8c0b0410 2026-02-25T15:01:12.720 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.716+0000 7f4d90f67640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d8c0b12b0 con 0x7f4d8c058da0 2026-02-25T15:01:12.720 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.716+0000 7f4d90f67640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d8c0b1590 con 0x7f4d8c1626a0 2026-02-25T15:01:12.720 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.716+0000 7f4d90f67640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:12.720 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.716+0000 7f4d90f67640 10 monclient(hunting): authenticate will time out at 2962.316075s 2026-02-25T15:01:12.720 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.716+0000 7f4d8a575640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f4d8c058da0 0x7f4d8c1647a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:48888/0 (socket says 10.20.193.90:48888) 2026-02-25T15:01:12.720 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.716+0000 7f4d8a575640 1 -- 10.20.193.90:0/3410621480 learned_addr learned my addr 10.20.193.90:0/3410621480 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:12.720 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.718+0000 7f4d89573640 1 -- 10.20.193.90:0/3410621480 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2852238451 0 0) 0x7f4d8c1655c0 con 0x7f4d8c0b0410 2026-02-25T15:01:12.721 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.718+0000 7f4d89573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:12.721 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.718+0000 7f4d89573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:12.721 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.718+0000 7f4d89573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:12.721 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.718+0000 7f4d89573640 10 monclient(hunting): my global_id is 4221 2026-02-25T15:01:12.721 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.718+0000 7f4d89573640 1 -- 10.20.193.90:0/3410621480 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d78003610 con 0x7f4d8c0b0410 2026-02-25T15:01:12.721 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.718+0000 7f4d89573640 1 -- 10.20.193.90:0/3410621480 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2362333274 0 0) 0x7f4d8c0b12b0 con 0x7f4d8c058da0 2026-02-25T15:01:12.721 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.718+0000 7f4d89573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:12.721 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.718+0000 7f4d89573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:12.721 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.718+0000 7f4d89573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:12.721 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.718+0000 7f4d89573640 10 monclient(hunting): my global_id is 4231 2026-02-25T15:01:12.721 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.718+0000 7f4d89573640 1 -- 10.20.193.90:0/3410621480 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d8c1655c0 con 0x7f4d8c058da0 2026-02-25T15:01:12.725 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d89573640 1 -- 10.20.193.90:0/3410621480 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 349491684 0 0) 0x7f4d8c1655c0 con 0x7f4d8c058da0 2026-02-25T15:01:12.725 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d89573640 1 -- 10.20.193.90:0/3410621480 >> v1:10.20.193.94:6789/0 conn(0x7f4d8c1626a0 legacy=0x7f4d8c1c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:12.725 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d89573640 1 -- 10.20.193.90:0/3410621480 >> v1:10.20.193.90:6789/0 conn(0x7f4d8c0b0410 legacy=0x7f4d8c164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:12.725 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d89573640 10 monclient: _finish_hunting 0 2026-02-25T15:01:12.725 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d89573640 1 monclient: found mon.noname-b 2026-02-25T15:01:12.725 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d89573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:12.725 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d89573640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:12.725 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d89573640 1 -- 10.20.193.90:0/3410621480 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c0b1870 con 0x7f4d8c058da0 2026-02-25T15:01:12.725 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d89573640 10 monclient: _finish_auth 0 2026-02-25T15:01:12.726 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d89573640 10 monclient: _check_auth_tickets 2026-02-25T15:01:12.726 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d89573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:12.726 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d89573640 1 -- 10.20.193.90:0/3410621480 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4d80002df0 con 0x7f4d8c058da0 2026-02-25T15:01:12.726 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d89573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:12.726 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d89573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:12.726 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d89573640 10 monclient: dump: 2026-02-25T15:01:12.726 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:12.726 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:12.726 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:12.726 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:12.726 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:12.726 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:12.727 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:12.727 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:12.727 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:12.727 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:12.727 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d90f67640 5 monclient: authenticate success, global_id 4231 2026-02-25T15:01:12.727 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.720+0000 7f4d90f67640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:12.727 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d89573640 1 -- 10.20.193.90:0/3410621480 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4d80003000 con 0x7f4d8c058da0 2026-02-25T15:01:12.727 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d89573640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:12.727 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d89573640 1 -- 10.20.193.90:0/3410621480 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4d80003400 con 0x7f4d8c058da0 2026-02-25T15:01:12.727 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d89573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:12.727 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d89573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:12.728 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d89573640 10 monclient: dump: 2026-02-25T15:01:12.728 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:12.728 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:12.728 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:12.728 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:12.728 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:12.728 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:12.728 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:12.728 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:12.728 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:12.728 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:12.728 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d90f67640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:12.728 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d90f67640 4 set_mon_vals no callback set 2026-02-25T15:01:12.729 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d90f67640 10 monclient: shutdown 2026-02-25T15:01:12.729 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d90f67640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:12.729 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d90f67640 1 -- 10.20.193.90:0/3410621480 >> v1:10.20.193.93:6789/0 conn(0x7f4d8c058da0 legacy=0x7f4d8c1647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:12.729 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d90f67640 1 -- 10.20.193.90:0/3410621480 shutdown_connections 2026-02-25T15:01:12.729 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d90f67640 1 -- 10.20.193.90:0/3410621480 >> 10.20.193.90:0/3410621480 conn(0x7f4d8c0b65f0 msgr2=0x7f4d8c0b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:12.729 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d90f67640 1 -- 10.20.193.90:0/3410621480 shutdown_connections 2026-02-25T15:01:12.729 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d90f67640 1 -- 10.20.193.90:0/3410621480 wait complete. 2026-02-25T15:01:12.729 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d90f67640 10 monclient: build_initial_monmap 2026-02-25T15:01:12.729 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d90f67640 10 monclient: monmap: 2026-02-25T15:01:12.729 INFO:teuthology.misc.health.trial090.stderr:epoch 0 2026-02-25T15:01:12.729 INFO:teuthology.misc.health.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:12.729 INFO:teuthology.misc.health.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:12.730 INFO:teuthology.misc.health.trial090.stderr:created 0.000000 2026-02-25T15:01:12.730 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:12.730 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 1 2026-02-25T15:01:12.730 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:12.730 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:12.730 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:12.730 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:12.730 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d90f67640 1 librados: starting msgr at 2026-02-25T15:01:12.730 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d90f67640 1 librados: starting objecter 2026-02-25T15:01:12.730 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.721+0000 7f4d90f67640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:12.730 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.722+0000 7f4d90f67640 1 Processor -- start 2026-02-25T15:01:12.730 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.722+0000 7f4d90f67640 1 -- start start 2026-02-25T15:01:12.730 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.722+0000 7f4d90f67640 1 librados: setting wanted keys 2026-02-25T15:01:12.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.722+0000 7f4d90f67640 1 librados: calling monclient init 2026-02-25T15:01:12.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.722+0000 7f4d90f67640 10 monclient: init 2026-02-25T15:01:12.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.722+0000 7f4d90f67640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:12.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.722+0000 7f4d90f67640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:12.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.722+0000 7f4d90f67640 10 monclient(hunting): picked mon.noname-c con 0x7f4d8c058da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:12.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.722+0000 7f4d90f67640 10 monclient(hunting): picked mon.noname-b con 0x7f4d8c0b0410 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:12.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.722+0000 7f4d90f67640 10 monclient(hunting): picked mon.noname-a con 0x7f4d8c1626a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:12.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.722+0000 7f4d90f67640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d8c14ea00 con 0x7f4d8c1626a0 2026-02-25T15:01:12.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.722+0000 7f4d90f67640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d8c05b970 con 0x7f4d8c0b0410 2026-02-25T15:01:12.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.722+0000 7f4d90f67640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d8c150d90 con 0x7f4d8c058da0 2026-02-25T15:01:12.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.722+0000 7f4d90f67640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:12.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.722+0000 7f4d90f67640 10 monclient(hunting): authenticate will time out at 2962.322474s 2026-02-25T15:01:12.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.723+0000 7f4d89d74640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f4d8c0b0410 0x7f4d8c14e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:48894/0 (socket says 10.20.193.90:48894) 2026-02-25T15:01:12.732 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.723+0000 7f4d89d74640 1 -- 10.20.193.90:0/410203572 learned_addr learned my addr 10.20.193.90:0/410203572 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:12.732 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.723+0000 7f4d72ffd640 10 client.?.objecter ms_handle_connect 0x7f4d8c1626a0 2026-02-25T15:01:12.732 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.723+0000 7f4d72ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:12.732 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.723+0000 7f4d72ffd640 10 client.?.objecter ms_handle_connect 0x7f4d8c0b0410 2026-02-25T15:01:12.732 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.723+0000 7f4d72ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:12.732 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.723+0000 7f4d72ffd640 10 client.?.objecter ms_handle_connect 0x7f4d8c058da0 2026-02-25T15:01:12.732 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.723+0000 7f4d72ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:12.735 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.724+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2998747939 0 0) 0x7f4d8c14ea00 con 0x7f4d8c1626a0 2026-02-25T15:01:12.735 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.724+0000 7f4d72ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:12.736 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.724+0000 7f4d72ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:12.736 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.724+0000 7f4d72ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:12.736 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.724+0000 7f4d72ffd640 10 monclient(hunting): my global_id is 4224 2026-02-25T15:01:12.736 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.724+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d58003610 con 0x7f4d8c1626a0 2026-02-25T15:01:12.736 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.724+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2163239681 0 0) 0x7f4d8c150d90 con 0x7f4d8c058da0 2026-02-25T15:01:12.736 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.724+0000 7f4d72ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:12.736 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.724+0000 7f4d72ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:12.736 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.724+0000 7f4d72ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:12.736 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.724+0000 7f4d72ffd640 10 monclient(hunting): my global_id is 4238 2026-02-25T15:01:12.736 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.724+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d8c14ea00 con 0x7f4d8c058da0 2026-02-25T15:01:12.736 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.725+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 625769114 0 0) 0x7f4d8c05b970 con 0x7f4d8c0b0410 2026-02-25T15:01:12.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.725+0000 7f4d72ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:12.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.725+0000 7f4d72ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:12.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.725+0000 7f4d72ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:12.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.725+0000 7f4d72ffd640 10 monclient(hunting): my global_id is 4234 2026-02-25T15:01:12.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.725+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d8c150d90 con 0x7f4d8c0b0410 2026-02-25T15:01:12.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 902700367 0 0) 0x7f4d8c14ea00 con 0x7f4d8c058da0 2026-02-25T15:01:12.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4d8c05b970 con 0x7f4d8c058da0 2026-02-25T15:01:12.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4d80002d00 con 0x7f4d8c058da0 2026-02-25T15:01:12.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:12.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:12.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:12.737 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:12.737 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:12.738 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:12.738 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:12.738 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:12.738 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:12.738 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:12.738 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:12.738 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:12.738 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:12.738 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:12.738 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3540090990 0 0) 0x7f4d58003610 con 0x7f4d8c1626a0 2026-02-25T15:01:12.738 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4d8c14ea00 con 0x7f4d8c1626a0 2026-02-25T15:01:12.738 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4d7c003270 con 0x7f4d8c1626a0 2026-02-25T15:01:12.739 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:12.739 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:12.739 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:12.739 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:12.739 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:12.739 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:12.739 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:12.739 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:12.739 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:12.739 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:12.739 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:12.739 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:12.739 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:12.740 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1613364683 0 0) 0x7f4d8c150d90 con 0x7f4d8c0b0410 2026-02-25T15:01:12.740 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4d58003610 con 0x7f4d8c0b0410 2026-02-25T15:01:12.740 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4d74003170 con 0x7f4d8c0b0410 2026-02-25T15:01:12.740 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:12.740 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:12.740 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.727+0000 7f4d72ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:12.740 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:12.740 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:12.740 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:12.740 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:12.740 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:12.740 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:12.741 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:12.741 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:12.741 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:12.741 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:12.741 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3830709014 0 0) 0x7f4d8c05b970 con 0x7f4d8c058da0 2026-02-25T15:01:12.741 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 >> v1:10.20.193.93:6789/0 conn(0x7f4d8c0b0410 legacy=0x7f4d8c14e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:12.741 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 >> v1:10.20.193.90:6789/0 conn(0x7f4d8c1626a0 legacy=0x7f4d8c150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:12.741 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d72ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:12.741 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d72ffd640 1 monclient: found mon.c 2026-02-25T15:01:12.741 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d72ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:12.741 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d72ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:12.741 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c05bc10 con 0x7f4d8c058da0 2026-02-25T15:01:12.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d72ffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:12.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d72ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:12.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d72ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:12.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d90f67640 5 monclient: authenticate success, global_id 4238 2026-02-25T15:01:12.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d90f67640 10 monclient: _renew_subs 2026-02-25T15:01:12.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d90f67640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:12.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d90f67640 1 -- 10.20.193.90:0/410203572 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8c05bef0 con 0x7f4d8c058da0 2026-02-25T15:01:12.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d90f67640 10 client.4238.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:12.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d90f67640 10 monclient: _renew_subs 2026-02-25T15:01:12.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d90f67640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:12.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d90f67640 1 -- 10.20.193.90:0/410203572 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4d8c05c4b0 con 0x7f4d8c058da0 2026-02-25T15:01:12.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d90f67640 1 librados: init done 2026-02-25T15:01:12.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d90f67640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:12.743 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d90f67640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:12.743 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d90f67640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:12.743 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.729+0000 7f4d90f67640 1 -- 10.20.193.90:0/410203572 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d8c0b6d40 con 0x7f4d8c058da0 2026-02-25T15:01:12.744 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.737+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4d80003000 con 0x7f4d8c058da0 2026-02-25T15:01:12.744 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.737+0000 7f4d72ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:12.744 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.737+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4d80004b80 con 0x7f4d8c058da0 2026-02-25T15:01:12.744 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.737+0000 7f4d72ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:12.744 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.737+0000 7f4d72ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:12.745 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.737+0000 7f4d72ffd640 10 monclient: dump: 2026-02-25T15:01:12.745 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:12.745 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:12.745 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:12.745 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:12.745 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:12.745 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:12.745 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:12.745 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:12.745 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:12.745 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:12.745 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.737+0000 7f4d88d72640 4 set_mon_vals no callback set 2026-02-25T15:01:12.746 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.737+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 4) ==== 103168+0+0 (unknown 1070002493 0 0) 0x7f4d8001e030 con 0x7f4d8c058da0 2026-02-25T15:01:12.746 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.738+0000 7f4d89d74640 10 monclient: get_auth_request con 0x7f4d5807a010 auth_method 0 2026-02-25T15:01:12.746 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.738+0000 7f4d72ffd640 10 client.4238.objecter ms_handle_connect 0x7f4d5807a010 2026-02-25T15:01:12.746 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.739+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(18..18 src has 1..18) ==== 2560+0+0 (unknown 1923581746 0 0) 0x7f4d8005aaf0 con 0x7f4d8c058da0 2026-02-25T15:01:12.746 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.739+0000 7f4d72ffd640 10 client.4238.objecter ms_dispatch2 0x7f4d8c000b90 osd_map(18..18 src has 1..18) 2026-02-25T15:01:12.746 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.739+0000 7f4d72ffd640 3 client.4238.objecter handle_osd_map got epochs [18,18] > 0 2026-02-25T15:01:12.746 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.739+0000 7f4d72ffd640 3 client.4238.objecter handle_osd_map decoding full epoch 18 2026-02-25T15:01:12.746 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.739+0000 7f4d72ffd640 20 client.4238.objecter dump_active .. 0 homeless 2026-02-25T15:01:12.826 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.825+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f4d8005f260 con 0x7f4d8c058da0 2026-02-25T15:01:12.826 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.825+0000 7f4d72ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:12.826 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.825+0000 7f4d72ffd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:12.937 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.935+0000 7f4d90f67640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:12.937 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.935+0000 7f4d90f67640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:12.937 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.935+0000 7f4d90f67640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:12.937 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.935+0000 7f4d90f67640 1 -- 10.20.193.90:0/410203572 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d8c0b6a60 con 0x7f4d8c058da0 2026-02-25T15:01:12.945 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.943+0000 7f4d72ffd640 1 -- 10.20.193.90:0/410203572 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12877 (unknown 3338201251 0 373084174) 0x7f4d8005eb60 con 0x7f4d8c058da0 2026-02-25T15:01:12.945 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.943+0000 7f4d72ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:12.945 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.943+0000 7f4d72ffd640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:12.945 INFO:teuthology.misc.health.trial090.stdout: 2026-02-25T15:01:12.945 INFO:teuthology.misc.health.trial090.stdout:{"epoch":18,"fsid":"63c679b5-d219-4797-8adb-7340f3fd6819","created":"2026-02-25T15:00:56.256818+0000","modified":"2026-02-25T15:01:02.711189+0000","last_up_change":"0.000000","last_in_change":"2026-02-25T15:01:02.711189+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ad97006e-f5c3-4e36-97a0-7fa2b250344d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"25b02f30-96b9-43a3-9691-f06a1292e36a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aeed908b-d483-498b-83ba-632e51dd05e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c7a30346-685f-4bd2-acaa-00b98ad3284c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d6c97301-55cc-4579-bc92-cb240c5ebe24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e3117058-2f0c-466b-a302-061fc4c120fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f4b4a3ce-0a0b-4386-a433-dcbf65558ecf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3337e549-6b6e-444e-997b-18a19beb5640","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"b292f7c4-9a41-4e48-876c-29281c5ec73b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"954a7cbb-2f06-4cb9-ae68-380ac58a9f21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"95ed1c74-8ca5-4317-9ef7-fc657b3cda70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"8f617b59-e8d1-42fb-8787-64e8c2117954","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"4bad4371-2ca7-4023-8406-0275011eda9c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"5d1ca952-93a5-43f3-9780-8bc85f9ed3d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"50853d20-2756-4720-b7aa-3020ded29189","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"4445eda7-fb77-4f2c-8fda-77667df22b47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T15:01:12.948 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.946+0000 7f4d90f67640 10 librados: watch_flush enter 2026-02-25T15:01:12.949 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.946+0000 7f4d90f67640 10 librados: watch_flush exit 2026-02-25T15:01:12.949 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.946+0000 7f4d90f67640 20 client.4238.objecter shutdown clearing up homeless session... 2026-02-25T15:01:12.949 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.946+0000 7f4d90f67640 10 client.4238.objecter successfully canceled tick 2026-02-25T15:01:12.949 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.946+0000 7f4d90f67640 1 -- 10.20.193.90:0/410203572 >> v1:10.20.193.189:6800/3647375982 conn(0x7f4d5807a010 legacy=0x7f4d5809a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:12.949 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.946+0000 7f4d90f67640 10 monclient: shutdown 2026-02-25T15:01:12.949 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.946+0000 7f4d90f67640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:12.949 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.946+0000 7f4d90f67640 1 -- 10.20.193.90:0/410203572 >> v1:10.20.193.94:6789/0 conn(0x7f4d8c058da0 legacy=0x7f4d8c0a8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:12.949 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.946+0000 7f4d90f67640 1 -- 10.20.193.90:0/410203572 shutdown_connections 2026-02-25T15:01:12.949 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.947+0000 7f4d90f67640 1 -- 10.20.193.90:0/410203572 >> 10.20.193.90:0/410203572 conn(0x7f4d8c0b65f0 msgr2=0x7f4d8c0afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:12.949 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.947+0000 7f4d90f67640 1 -- 10.20.193.90:0/410203572 shutdown_connections 2026-02-25T15:01:12.949 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.947+0000 7f4d90f67640 1 -- 10.20.193.90:0/410203572 wait complete. 2026-02-25T15:01:12.950 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:12.947+0000 7f4d90f67640 1 librados: shutdown 2026-02-25T15:01:12.956 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-25T15:01:13.085 INFO:tasks.ceph.osd.2.trial094.stderr:--00:00:00:10.277 45092-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:13.086 INFO:tasks.ceph.osd.10.trial094.stderr:--00:00:00:10.274 45093-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:13.123 INFO:tasks.ceph.osd.15.trial189.stderr:--00:00:00:10.297 45483-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:13.123 INFO:tasks.ceph.osd.11.trial189.stderr:--00:00:00:10.298 45482-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:13.124 INFO:tasks.ceph.osd.5.trial093.stderr:--00:00:00:10.326 45235-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:13.125 INFO:tasks.ceph.osd.1.trial093.stderr:--00:00:00:10.329 45234-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:13.197 INFO:tasks.ceph.osd.8.trial090.stderr:--00:00:00:10.407 45913-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:13.198 INFO:tasks.ceph.osd.0.trial090.stderr:--00:00:00:10.413 45910-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:13.198 INFO:tasks.ceph.osd.4.trial090.stderr:--00:00:00:10.411 45911-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:13.211 INFO:tasks.ceph.osd.7.trial189.stderr:--00:00:00:10.386 45481-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:13.213 INFO:tasks.ceph.osd.14.trial094.stderr:--00:00:00:10.397 45095-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:13.217 INFO:tasks.ceph.osd.6.trial094.stderr:--00:00:00:10.404 45094-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:13.219 INFO:tasks.ceph.osd.12.trial090.stderr:--00:00:00:10.427 45917-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:13.369 INFO:tasks.ceph.osd.9.trial093.stderr:--00:00:00:10.568 45236-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:13.435 INFO:tasks.ceph.osd.3.trial189.stderr:--00:00:00:10.611 45480-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:13.460 INFO:tasks.ceph.osd.13.trial093.stderr:--00:00:00:10.656 45237-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/4f/df6491843300f77b7abd294d2e264b9099e4c0.debug 2026-02-25T15:01:15.216 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:12.418 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.222 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:12.411 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.228 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:12.420 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.238 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:12.414 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.242 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:12.446 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.252 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:12.426 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.286 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:12.489 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.300 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:12.476 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.301 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:12.493 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.301 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:12.490 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.305 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:12.508 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.318 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:12.522 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.318 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:12.494 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.325 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:12.499 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.325 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:12.501 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.325 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:12.518 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.329 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:12.540 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.329 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:12.518 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.333 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:12.547 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.338 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:12.554 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.345 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:12.547 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.353 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:12.529 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.353 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:12.537 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.354 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:12.546 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.358 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:12.532 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.361 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:12.566 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.364 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:12.551 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.369 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:12.558 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.371 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:12.579 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.379 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:12.582 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.388 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:12.593 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.389 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:12.564 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.394 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:12.570 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.396 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:12.607 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.398 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:12.590 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.400 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:12.574 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.401 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:12.615 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.406 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:12.595 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.408 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:12.611 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.414 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:12.629 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.420 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:12.604 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.423 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:12.626 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.425 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:12.601 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.426 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:12.619 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.427 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:12.631 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.428 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:12.616 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.431 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:12.620 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.431 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:12.645 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.433 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:12.607 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.437 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:12.612 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.437 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:12.648 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.438 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:12.637 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.442 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:12.634 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.445 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:12.628 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.446 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:12.634 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.448 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:12.656 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.452 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:12.628 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.452 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:12.666 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.454 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:12.670 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.455 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:12.631 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.463 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:12.637 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.463 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:12.674 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.466 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:12.653 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.474 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:12.682 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.475 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:12.680 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.479 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:12.653 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.482 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:12.666 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.482 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:12.696 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.483 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:12.698 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.486 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:12.662 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.490 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:12.701 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.491 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:12.695 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.496 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:12.683 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.499 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:12.707 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.501 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:12.700 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.506 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:12.722 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.508 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:12.692 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.508 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:12.722 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.517 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:12.728 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.520 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:12.707 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.523 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:12.737 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.524 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:12.700 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.525 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:12.733 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.528 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:12.724 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.529 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:12.745 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.530 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:12.705 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.531 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:12.714 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.531 INFO:tasks.ceph.osd.5.trial093.stderr:2026-02-25T15:01:15.525+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.532 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:12.743 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.534 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:12.734 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.539 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:12.715 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.545 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:12.761 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.546 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:12.733 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.547 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:12.730 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.552 INFO:tasks.ceph.osd.2.trial094.stderr:2026-02-25T15:01:15.548+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.561 INFO:tasks.ceph.osd.10.trial094.stderr:2026-02-25T15:01:15.556+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.561 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:12.749 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.565 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:12.773 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.575 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:12.774 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.580 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:12.788 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.582 INFO:tasks.ceph.osd.11.trial189.stderr:2026-02-25T15:01:15.578+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.601 INFO:tasks.ceph.osd.15.trial189.stderr:2026-02-25T15:01:15.597+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.601 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:12.776 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.601 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:12.797 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.602 INFO:tasks.ceph.osd.1.trial093.stderr:2026-02-25T15:01:15.596+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.613 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:12.813 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.633 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:12.830 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.637 INFO:tasks.ceph.osd.8.trial090.stderr:2026-02-25T15:01:15.634+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.638 INFO:tasks.ceph.osd.4.trial090.stderr:2026-02-25T15:01:15.634+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.640 INFO:tasks.ceph.osd.8.trial090.stderr:2026-02-25T15:01:15.638+0000 a75f640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6800/0: (98) Address already in use 2026-02-25T15:01:15.642 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:12.817 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.653 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:12.853 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.659 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:12.855 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.661 INFO:tasks.ceph.osd.14.trial094.stderr:2026-02-25T15:01:15.657+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.662 INFO:tasks.ceph.osd.0.trial090.stderr:2026-02-25T15:01:15.658+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.664 INFO:tasks.ceph.osd.7.trial189.stderr:2026-02-25T15:01:15.661+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.668 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:12.867 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.670 INFO:tasks.ceph.osd.6.trial094.stderr:2026-02-25T15:01:15.666+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.676 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:12.852 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.695 INFO:tasks.ceph.osd.12.trial090.stderr:2026-02-25T15:01:15.691+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.710 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:12.906 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.712 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:12.887 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.742 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:12.918 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.753 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:12.949 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.758 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:12.933 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.769 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:12.965 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.795 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:12.997 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.805 INFO:tasks.ceph.osd.9.trial093.stderr:2026-02-25T15:01:15.799+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.824 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:13.016 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.838 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:13.027 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.854 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:13.030 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.864 INFO:tasks.ceph.osd.13.trial093.stderr:2026-02-25T15:01:15.858+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.868 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:13.071 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.870 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:13.074 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.879 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:13.054 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.899 INFO:tasks.ceph.osd.3.trial189.stderr:2026-02-25T15:01:15.895+0000 787d4c0 -1 Falling back to public interface 2026-02-25T15:01:15.904 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:13.096 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.907 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:13.121 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.908 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:13.118 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.929 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:13.113 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.931 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:13.120 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.936 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:13.152 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.940 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:13.115 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.945 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:13.121 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.948 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:13.151 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.950 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:13.137 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.956 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:13.148 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.961 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:13.165 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.968 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:13.176 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.972 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:13.146 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.977 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:13.169 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.978 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:13.167 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.990 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:13.192 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:15.992 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:13.167 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.008 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:13.218 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.009 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:13.222 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.014 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:13.198 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.018 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:13.206 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.018 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:13.210 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.029 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:13.231 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.030 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:13.206 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.032 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:13.236 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.037 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:13.213 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.037 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:13.225 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.040 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:13.214 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.040 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:13.256 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.045 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:13.233 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.058 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:13.250 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.064 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:13.240 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.066 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:13.279 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.066 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:13.274 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.067 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:13.255 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.068 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:13.242 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.068 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:13.272 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.070 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:13.254 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.072 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:13.272 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.073 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:13.275 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.084 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:13.276 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.086 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:13.296 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.089 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:13.304 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.091 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:13.305 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.094 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:13.270 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.099 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:13.287 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.103 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:13.277 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.107 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:13.290 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.107 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:13.296 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.109 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:13.284 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.111 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:13.319 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.113 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:13.329 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.116 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:13.308 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.117 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:13.331 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.118 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:13.322 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.118 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:13.320 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.119 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:13.295 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.128 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:13.315 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.132 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:13.342 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.136 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:13.319 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.136 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:13.325 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.136 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:13.328 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.137 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:13.310 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.137 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:13.333 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.139 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:13.347 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.141 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:13.355 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.142 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:13.317 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.144 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:13.360 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.156 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:13.332 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.159 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:13.342 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.159 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:13.346 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.162 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:13.361 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.162 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:13.351 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.168 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:13.370 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.168 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:13.372 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.169 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:13.383 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.170 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:13.377 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.171 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:13.346 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.171 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:13.345 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.175 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:13.385 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.175 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:13.391 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.180 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:13.356 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.181 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:13.356 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.181 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:13.365 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.196 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:13.370 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.197 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:13.411 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.198 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:13.386 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.202 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:13.410 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.209 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:13.425 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.218 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:13.422 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.218 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:13.420 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.221 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:13.397 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.222 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:13.432 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.222 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:13.419 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.225 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:13.408 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.227 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:13.427 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.229 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:13.443 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.229 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:13.437 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.233 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:13.407 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.233 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:13.421 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.237 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:13.453 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.248 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:13.423 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.253 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:13.461 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.258 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:13.468 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.259 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:13.451 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.265 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:13.481 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.266 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:13.449 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.269 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:13.457 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.273 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:13.473 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.274 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:13.478 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.276 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:13.452 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.282 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:13.490 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.284 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:13.473 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.291 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:13.501 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.306 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:13.494 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.309 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:13.485 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.314 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:13.506 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.315 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:13.515 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.316 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:13.520 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.317 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:13.513 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.321 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:13.496 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.327 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:13.515 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.329 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:13.505 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.338 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:13.549 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.339 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:13.531 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.346 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:13.545 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.346 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:13.560 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.351 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:13.525 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.356 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:13.558 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.359 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:13.547 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.360 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:13.535 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.363 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:13.539 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.364 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:13.560 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.364 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:13.540 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.374 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:13.566 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.379 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:13.571 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.382 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:13.582 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.384 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:13.560 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.385 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:13.600 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.390 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:13.573 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.392 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:13.567 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.398 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:13.612 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.402 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:13.610 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.404 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:13.578 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.406 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:13.581 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.406 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:13.595 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.408 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:13.604 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.411 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:13.613 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.411 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:13.600 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.411 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:13.587 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.413 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:13.613 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.426 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:13.601 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.428 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:13.616 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.432 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:13.646 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.440 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:13.614 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.443 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:13.626 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.445 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:13.647 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.446 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:13.646 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.448 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:13.652 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.450 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:13.665 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.452 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:13.648 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.457 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:13.665 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.459 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:13.672 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.464 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:13.678 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.471 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:13.646 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.480 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:13.664 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.481 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:13.696 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.482 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:13.693 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.483 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:13.657 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.488 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:13.675 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.488 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:13.662 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.489 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:13.692 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.495 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:13.697 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.498 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:13.706 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.501 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:13.705 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.502 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:13.698 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.506 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:13.681 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.508 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:13.683 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.509 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:13.693 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.514 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:13.698 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.517 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:13.732 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.523 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:13.710 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.523 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:13.738 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.532 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:13.737 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.534 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:13.730 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.538 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:13.713 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.541 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:13.749 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.546 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:13.754 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.552 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:13.762 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.554 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:13.730 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.555 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:13.742 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.560 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:13.748 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.573 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:13.777 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.578 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:13.782 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.582 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:13.778 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.582 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:13.782 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.585 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:13.795 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.587 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:13.763 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.615 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:13.790 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.619 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:13.829 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.620 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:13.796 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.625 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:13.824 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.625 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:13.835 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.658 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:13.857 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.671 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:13.846 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.693 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:13.892 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.698 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:13.897 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.717 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:13.913 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.728 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:13.904 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.757 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:13.933 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.769 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:13.965 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.787 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:13.962 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.792 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:13.967 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.800 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:13.996 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.825 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:14.021 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:16.831 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:14.027 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:18.648 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:15.840 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:18.696 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:15.884 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:18.721 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:15.897 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:18.724 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:15.938 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:18.760 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:15.963 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:18.776 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:15.960 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:18.789 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:16.004 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:18.800 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:15.974 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:18.817 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:16.004 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:18.817 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:16.025 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:18.850 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:16.054 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:18.922 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:16.132 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:18.940 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:16.116 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:18.952 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:16.151 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:19.093 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:16.269 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:19.095 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:16.291 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:20.567 DEBUG:teuthology.orchestra.run.trial090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-25T15:01:20.628 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.627+0000 7f8c44a16640 10 monclient: get_monmap_and_config 2026-02-25T15:01:20.628 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.627+0000 7f8c44a16640 10 monclient: build_initial_monmap 2026-02-25T15:01:20.628 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.627+0000 7f8c44a16640 10 monclient: monmap: 2026-02-25T15:01:20.628 INFO:teuthology.misc.health.trial090.stderr:epoch 0 2026-02-25T15:01:20.628 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:20.628 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:01:20.628426+0000 2026-02-25T15:01:20.628 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:01:20.628426+0000 2026-02-25T15:01:20.629 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:20.629 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:20.629 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:20.629 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:20.629 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:20.629 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:20.629 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.628+0000 7f8c44a16640 1 Processor -- start 2026-02-25T15:01:20.629 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.628+0000 7f8c44a16640 1 -- start start 2026-02-25T15:01:20.629 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.628+0000 7f8c44a16640 10 monclient: init 2026-02-25T15:01:20.629 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.628+0000 7f8c44a16640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:20.629 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.628+0000 7f8c44a16640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:20.629 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.628+0000 7f8c44a16640 10 monclient(hunting): picked mon.noname-c con 0x7f8c40055bf0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:20.629 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.628+0000 7f8c44a16640 10 monclient(hunting): picked mon.noname-a con 0x7f8c40164040 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:20.630 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.628+0000 7f8c44a16640 10 monclient(hunting): picked mon.noname-b con 0x7f8c40055fc0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:20.630 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.628+0000 7f8c44a16640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c400b3a00 con 0x7f8c40164040 2026-02-25T15:01:20.630 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.628+0000 7f8c44a16640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c400b3c70 con 0x7f8c40055fc0 2026-02-25T15:01:20.630 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.628+0000 7f8c44a16640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c400b3f50 con 0x7f8c40055bf0 2026-02-25T15:01:20.630 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.628+0000 7f8c44a16640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:20.630 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.628+0000 7f8c44a16640 10 monclient(hunting): authenticate will time out at 2970.228865s 2026-02-25T15:01:20.630 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.629+0000 7f8c3dd74640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f8c40164040 0x7f8c40056690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:35740/0 (socket says 10.20.193.90:35740) 2026-02-25T15:01:20.630 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.629+0000 7f8c3dd74640 1 -- 10.20.193.90:0/1634117434 learned_addr learned my addr 10.20.193.90:0/1634117434 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:20.631 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.631+0000 7f8c3d573640 1 -- 10.20.193.90:0/1634117434 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1359015401 0 0) 0x7f8c400b3a00 con 0x7f8c40164040 2026-02-25T15:01:20.631 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.631+0000 7f8c3d573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:20.631 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.631+0000 7f8c3d573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:20.631 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.631+0000 7f8c3d573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:20.632 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.631+0000 7f8c3d573640 10 monclient(hunting): my global_id is 4275 2026-02-25T15:01:20.632 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.631+0000 7f8c3d573640 1 -- 10.20.193.90:0/1634117434 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c2c003610 con 0x7f8c40164040 2026-02-25T15:01:20.632 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.631+0000 7f8c3d573640 1 -- 10.20.193.90:0/1634117434 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1488598848 0 0) 0x7f8c400b3f50 con 0x7f8c40055bf0 2026-02-25T15:01:20.632 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.631+0000 7f8c3d573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:20.632 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.631+0000 7f8c3d573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:20.632 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.631+0000 7f8c3d573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:20.632 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.631+0000 7f8c3d573640 10 monclient(hunting): my global_id is 4289 2026-02-25T15:01:20.632 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.631+0000 7f8c3d573640 1 -- 10.20.193.90:0/1634117434 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c400b3a00 con 0x7f8c40055bf0 2026-02-25T15:01:20.633 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.632+0000 7f8c3d573640 1 -- 10.20.193.90:0/1634117434 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2939161290 0 0) 0x7f8c400b3c70 con 0x7f8c40055fc0 2026-02-25T15:01:20.633 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.632+0000 7f8c3d573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:20.633 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.632+0000 7f8c3d573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:20.633 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.632+0000 7f8c3d573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:20.633 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.632+0000 7f8c3d573640 10 monclient(hunting): my global_id is 4285 2026-02-25T15:01:20.633 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.632+0000 7f8c3d573640 1 -- 10.20.193.90:0/1634117434 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c400b3f50 con 0x7f8c40055fc0 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c3d573640 1 -- 10.20.193.90:0/1634117434 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1849131387 0 0) 0x7f8c2c003610 con 0x7f8c40164040 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c3d573640 1 -- 10.20.193.90:0/1634117434 >> v1:10.20.193.94:6789/0 conn(0x7f8c40055bf0 legacy=0x7f8c400b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c3d573640 1 -- 10.20.193.90:0/1634117434 >> v1:10.20.193.93:6789/0 conn(0x7f8c40055fc0 legacy=0x7f8c400b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c3d573640 10 monclient: _finish_hunting 0 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c3d573640 1 monclient: found mon.noname-a 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c3d573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c3d573640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c3d573640 1 -- 10.20.193.90:0/1634117434 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c400b4230 con 0x7f8c40164040 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c3d573640 10 monclient: _finish_auth 0 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c3d573640 10 monclient: _check_auth_tickets 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c3d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c3d573640 1 -- 10.20.193.90:0/1634117434 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8c28002df0 con 0x7f8c40164040 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c44a16640 5 monclient: authenticate success, global_id 4275 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c44a16640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c3d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c3d573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c3d573640 10 monclient: dump: 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:20.634 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.633+0000 7f8c44a16640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c3d573640 1 -- 10.20.193.90:0/1634117434 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8c28002fc0 con 0x7f8c40164040 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c3d573640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c3d573640 1 -- 10.20.193.90:0/1634117434 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8c28003400 con 0x7f8c40164040 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c3d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c3d573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c3d573640 10 monclient: dump: 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c44a16640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c44a16640 4 set_mon_vals no callback set 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c44a16640 10 monclient: shutdown 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c44a16640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c44a16640 1 -- 10.20.193.90:0/1634117434 >> v1:10.20.193.90:6789/0 conn(0x7f8c40164040 legacy=0x7f8c40056690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c44a16640 1 -- 10.20.193.90:0/1634117434 shutdown_connections 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c44a16640 1 -- 10.20.193.90:0/1634117434 >> 10.20.193.90:0/1634117434 conn(0x7f8c400b7bb0 msgr2=0x7f8c400b7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c44a16640 1 -- 10.20.193.90:0/1634117434 shutdown_connections 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c44a16640 1 -- 10.20.193.90:0/1634117434 wait complete. 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c44a16640 10 monclient: build_initial_monmap 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.634+0000 7f8c44a16640 10 monclient: monmap: 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:epoch 0 2026-02-25T15:01:20.635 INFO:teuthology.misc.health.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:created 0.000000 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 1 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 1 librados: starting msgr at 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 1 librados: starting objecter 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 1 Processor -- start 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 1 -- start start 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 1 librados: setting wanted keys 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 1 librados: calling monclient init 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 10 monclient: init 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 10 monclient(hunting): picked mon.noname-b con 0x7f8c40055bf0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 10 monclient(hunting): picked mon.noname-c con 0x7f8c40055fc0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 10 monclient(hunting): picked mon.noname-a con 0x7f8c40164040 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c400b6d00 con 0x7f8c40164040 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c40129890 con 0x7f8c40055bf0 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c40129b70 con 0x7f8c40055fc0 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:20.636 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c44a16640 10 monclient(hunting): authenticate will time out at 2970.235586s 2026-02-25T15:01:20.637 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c3e575640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f8c40055bf0 0x7f8c400b65f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46734/0 (socket says 10.20.193.90:46734) 2026-02-25T15:01:20.637 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.635+0000 7f8c3e575640 1 -- 10.20.193.90:0/3004805855 learned_addr learned my addr 10.20.193.90:0/3004805855 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:20.637 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.636+0000 7f8c26ffd640 10 client.?.objecter ms_handle_connect 0x7f8c40055bf0 2026-02-25T15:01:20.637 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.636+0000 7f8c26ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:20.637 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.636+0000 7f8c26ffd640 10 client.?.objecter ms_handle_connect 0x7f8c40055fc0 2026-02-25T15:01:20.637 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.636+0000 7f8c26ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:20.637 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.637+0000 7f8c26ffd640 10 client.?.objecter ms_handle_connect 0x7f8c40164040 2026-02-25T15:01:20.637 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.637+0000 7f8c26ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:20.638 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.637+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3581367 0 0) 0x7f8c40129890 con 0x7f8c40055bf0 2026-02-25T15:01:20.638 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.637+0000 7f8c26ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:20.638 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.637+0000 7f8c26ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:20.638 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.637+0000 7f8c26ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:20.638 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.637+0000 7f8c26ffd640 10 monclient(hunting): my global_id is 4288 2026-02-25T15:01:20.638 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.637+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c0c003610 con 0x7f8c40055bf0 2026-02-25T15:01:20.638 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.638+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2488257953 0 0) 0x7f8c40129b70 con 0x7f8c40055fc0 2026-02-25T15:01:20.638 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.638+0000 7f8c26ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:20.638 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.638+0000 7f8c26ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:20.639 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.638+0000 7f8c26ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:20.639 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.638+0000 7f8c26ffd640 10 monclient(hunting): my global_id is 4292 2026-02-25T15:01:20.639 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.638+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c40129890 con 0x7f8c40055fc0 2026-02-25T15:01:20.639 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.638+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1494372124 0 0) 0x7f8c400b6d00 con 0x7f8c40164040 2026-02-25T15:01:20.639 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.638+0000 7f8c26ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:20.639 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.638+0000 7f8c26ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:20.639 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.638+0000 7f8c26ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:20.639 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.638+0000 7f8c26ffd640 10 monclient(hunting): my global_id is 4278 2026-02-25T15:01:20.639 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.638+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c40129b70 con 0x7f8c40164040 2026-02-25T15:01:20.640 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.639+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3609558548 0 0) 0x7f8c0c003610 con 0x7f8c40055bf0 2026-02-25T15:01:20.640 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.639+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8c400b6d00 con 0x7f8c40055bf0 2026-02-25T15:01:20.640 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.639+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8c34003190 con 0x7f8c40055bf0 2026-02-25T15:01:20.640 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.639+0000 7f8c26ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:20.640 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.639+0000 7f8c26ffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:20.640 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.639+0000 7f8c26ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:20.640 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.639+0000 7f8c26ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.640+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3825109123 0 0) 0x7f8c40129890 con 0x7f8c40055fc0 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.640+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8c0c003610 con 0x7f8c40055fc0 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.640+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8c28003310 con 0x7f8c40055fc0 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.640+0000 7f8c26ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.640+0000 7f8c26ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.640+0000 7f8c26ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.640+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 871329441 0 0) 0x7f8c40129b70 con 0x7f8c40164040 2026-02-25T15:01:20.641 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.640+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8c40129890 con 0x7f8c40164040 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.640+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8c30003410 con 0x7f8c40164040 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.640+0000 7f8c26ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.640+0000 7f8c26ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.640+0000 7f8c26ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3184560068 0 0) 0x7f8c400b6d00 con 0x7f8c40055bf0 2026-02-25T15:01:20.642 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 >> v1:10.20.193.94:6789/0 conn(0x7f8c40055fc0 legacy=0x7f8c4012c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 >> v1:10.20.193.90:6789/0 conn(0x7f8c40164040 legacy=0x7f8c401529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c26ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c26ffd640 1 monclient: found mon.b 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c26ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c26ffd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c40129e50 con 0x7f8c40055bf0 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c26ffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c26ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c26ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c44a16640 5 monclient: authenticate success, global_id 4288 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c44a16640 10 monclient: _renew_subs 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c44a16640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c44a16640 1 -- 10.20.193.90:0/3004805855 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4012a130 con 0x7f8c40055bf0 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.641+0000 7f8c44a16640 10 client.4288.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.642+0000 7f8c44a16640 10 monclient: _renew_subs 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.642+0000 7f8c44a16640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.642+0000 7f8c44a16640 1 -- 10.20.193.90:0/3004805855 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f8c4012cd10 con 0x7f8c40055bf0 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.642+0000 7f8c44a16640 1 librados: init done 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.642+0000 7f8c44a16640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.642+0000 7f8c44a16640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.642+0000 7f8c44a16640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:20.643 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.642+0000 7f8c44a16640 1 -- 10.20.193.90:0/3004805855 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c4012a650 con 0x7f8c40055bf0 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.736+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8c340034e0 con 0x7f8c40055bf0 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.736+0000 7f8c26ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.736+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8c34003b50 con 0x7f8c40055bf0 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.736+0000 7f8c26ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.736+0000 7f8c26ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.736+0000 7f8c26ffd640 10 monclient: dump: 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:20.737 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:20.738 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.736+0000 7f8c3cd72640 4 set_mon_vals no callback set 2026-02-25T15:01:20.738 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.737+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 4) ==== 103168+0+0 (unknown 1070002493 0 0) 0x7f8c34003d30 con 0x7f8c40055bf0 2026-02-25T15:01:20.739 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.738+0000 7f8c3dd74640 10 monclient: get_auth_request con 0x7f8c0c079f60 auth_method 0 2026-02-25T15:01:20.739 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.738+0000 7f8c26ffd640 10 client.4288.objecter ms_handle_connect 0x7f8c0c079f60 2026-02-25T15:01:20.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.741+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(18..18 src has 1..18) ==== 2560+0+0 (unknown 1923581746 0 0) 0x7f8c3405ad50 con 0x7f8c40055bf0 2026-02-25T15:01:20.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.741+0000 7f8c26ffd640 10 client.4288.objecter ms_dispatch2 0x7f8c40000b90 osd_map(18..18 src has 1..18) 2026-02-25T15:01:20.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.741+0000 7f8c26ffd640 3 client.4288.objecter handle_osd_map got epochs [18,18] > 0 2026-02-25T15:01:20.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.741+0000 7f8c26ffd640 3 client.4288.objecter handle_osd_map decoding full epoch 18 2026-02-25T15:01:20.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.741+0000 7f8c26ffd640 20 client.4288.objecter dump_active .. 0 homeless 2026-02-25T15:01:20.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.741+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f8c4012a650 con 0x7f8c40055bf0 2026-02-25T15:01:20.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.741+0000 7f8c26ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:20.742 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.741+0000 7f8c26ffd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:20.857 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.856+0000 7f8c44a16640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:20.857 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.856+0000 7f8c44a16640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:20.857 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.856+0000 7f8c44a16640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:20.857 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.856+0000 7f8c44a16640 1 -- 10.20.193.90:0/3004805855 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c4012cfa0 con 0x7f8c40055bf0 2026-02-25T15:01:20.865 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.864+0000 7f8c26ffd640 1 -- 10.20.193.90:0/3004805855 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12877 (unknown 3338201251 0 373084174) 0x7f8c3405f4c0 con 0x7f8c40055bf0 2026-02-25T15:01:20.865 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.864+0000 7f8c26ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:20.865 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.864+0000 7f8c26ffd640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:20.865 INFO:teuthology.misc.health.trial090.stdout: 2026-02-25T15:01:20.865 INFO:teuthology.misc.health.trial090.stdout:{"epoch":18,"fsid":"63c679b5-d219-4797-8adb-7340f3fd6819","created":"2026-02-25T15:00:56.256818+0000","modified":"2026-02-25T15:01:02.711189+0000","last_up_change":"0.000000","last_in_change":"2026-02-25T15:01:02.711189+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ad97006e-f5c3-4e36-97a0-7fa2b250344d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"25b02f30-96b9-43a3-9691-f06a1292e36a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aeed908b-d483-498b-83ba-632e51dd05e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c7a30346-685f-4bd2-acaa-00b98ad3284c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d6c97301-55cc-4579-bc92-cb240c5ebe24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e3117058-2f0c-466b-a302-061fc4c120fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f4b4a3ce-0a0b-4386-a433-dcbf65558ecf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3337e549-6b6e-444e-997b-18a19beb5640","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"b292f7c4-9a41-4e48-876c-29281c5ec73b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"954a7cbb-2f06-4cb9-ae68-380ac58a9f21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"95ed1c74-8ca5-4317-9ef7-fc657b3cda70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"8f617b59-e8d1-42fb-8787-64e8c2117954","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"4bad4371-2ca7-4023-8406-0275011eda9c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"5d1ca952-93a5-43f3-9780-8bc85f9ed3d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"50853d20-2756-4720-b7aa-3020ded29189","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"4445eda7-fb77-4f2c-8fda-77667df22b47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T15:01:20.868 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.867+0000 7f8c44a16640 10 librados: watch_flush enter 2026-02-25T15:01:20.868 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.867+0000 7f8c44a16640 10 librados: watch_flush exit 2026-02-25T15:01:20.868 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.867+0000 7f8c44a16640 20 client.4288.objecter shutdown clearing up homeless session... 2026-02-25T15:01:20.868 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.867+0000 7f8c44a16640 10 client.4288.objecter successfully canceled tick 2026-02-25T15:01:20.868 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.867+0000 7f8c44a16640 1 -- 10.20.193.90:0/3004805855 >> v1:10.20.193.189:6800/3647375982 conn(0x7f8c0c079f60 legacy=0x7f8c0c09a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:20.868 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.867+0000 7f8c44a16640 10 monclient: shutdown 2026-02-25T15:01:20.868 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.867+0000 7f8c44a16640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:20.868 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.867+0000 7f8c44a16640 1 -- 10.20.193.90:0/3004805855 >> v1:10.20.193.93:6789/0 conn(0x7f8c40055bf0 legacy=0x7f8c400b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:20.868 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.867+0000 7f8c44a16640 1 -- 10.20.193.90:0/3004805855 shutdown_connections 2026-02-25T15:01:20.868 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.867+0000 7f8c44a16640 1 -- 10.20.193.90:0/3004805855 >> 10.20.193.90:0/3004805855 conn(0x7f8c400b7bb0 msgr2=0x7f8c400b7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:20.868 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.867+0000 7f8c44a16640 1 -- 10.20.193.90:0/3004805855 shutdown_connections 2026-02-25T15:01:20.868 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.867+0000 7f8c44a16640 1 -- 10.20.193.90:0/3004805855 wait complete. 2026-02-25T15:01:20.868 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:20.867+0000 7f8c44a16640 1 librados: shutdown 2026-02-25T15:01:20.877 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-25T15:01:20.921 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:18.113 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:20.946 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:18.135 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:20.951 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:18.144 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:20.954 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:18.146 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:20.967 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:18.181 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:20.984 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:18.173 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:20.987 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:18.176 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:20.993 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:18.195 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.005 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:18.219 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.008 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:18.222 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.016 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:18.192 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.020 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:18.223 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.023 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:18.225 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.034 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:18.218 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.036 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:18.251 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.045 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:18.221 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.048 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:18.224 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.053 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:18.261 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.061 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:18.245 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.064 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:18.248 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.065 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:18.269 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.070 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:18.258 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.071 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:18.287 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.074 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:18.290 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.092 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:18.300 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.095 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:18.303 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.096 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:18.301 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.099 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:18.304 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.104 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:18.278 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.110 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:18.298 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.114 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:18.301 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.135 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:18.309 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.138 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:18.312 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.156 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:18.366 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.174 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:18.373 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.191 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:18.402 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.194 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:18.405 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.207 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:18.407 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.210 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:18.409 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.221 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:18.397 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.252 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:18.427 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.255 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:18.431 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.330 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:18.526 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.350 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:18.526 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.356 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:18.552 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.359 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:18.555 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.391 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:18.566 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:21.394 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:18.569 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:22.547 INFO:tasks.ceph.osd.2.trial094.stderr:==00:00:00:19.739 45092== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:22.585 INFO:tasks.ceph.osd.10.trial094.stderr:==00:00:00:19.774 45093== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:22.608 INFO:tasks.ceph.osd.4.trial090.stderr:==00:00:00:19.822 45911== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:22.622 INFO:tasks.ceph.osd.5.trial093.stderr:==00:00:00:19.824 45235== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:22.649 INFO:tasks.ceph.osd.0.trial090.stderr:==00:00:00:19.865 45910== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:22.671 INFO:tasks.ceph.osd.12.trial090.stderr:==00:00:00:19.879 45917== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:22.672 INFO:tasks.ceph.osd.14.trial094.stderr:==00:00:00:19.855 45095== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:22.683 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:19.859 45482== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:22.694 INFO:tasks.ceph.osd.1.trial093.stderr:==00:00:00:19.898 45234== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:22.708 INFO:tasks.ceph.osd.6.trial094.stderr:==00:00:00:19.895 45094== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:22.760 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:19.933 45483== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:22.778 INFO:tasks.ceph.osd.9.trial093.stderr:==00:00:00:19.977 45236== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:22.798 INFO:tasks.ceph.osd.8.trial090.stderr:==00:00:00:20.009 45913== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:22.894 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:20.070 45481== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:22.951 INFO:tasks.ceph.osd.13.trial093.stderr:==00:00:00:20.147 45237== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:23.014 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:20.189 45480== Warning: unimplemented fcntl command: 1038 2026-02-25T15:01:25.307 INFO:tasks.ceph.osd.2.trial094.stderr:2026-02-25T15:01:25.303+0000 787d4c0 -1 osd.2 0 log_to_monitors true 2026-02-25T15:01:25.350 INFO:tasks.ceph.osd.10.trial094.stderr:2026-02-25T15:01:25.347+0000 787d4c0 -1 osd.10 0 log_to_monitors true 2026-02-25T15:01:25.358 INFO:tasks.ceph.osd.4.trial090.stderr:2026-02-25T15:01:25.354+0000 787d4c0 -1 osd.4 0 log_to_monitors true 2026-02-25T15:01:25.372 INFO:tasks.ceph.osd.5.trial093.stderr:2026-02-25T15:01:25.367+0000 787d4c0 -1 osd.5 0 log_to_monitors true 2026-02-25T15:01:25.393 INFO:tasks.ceph.osd.0.trial090.stderr:2026-02-25T15:01:25.390+0000 787d4c0 -1 osd.0 0 log_to_monitors true 2026-02-25T15:01:25.443 INFO:tasks.ceph.osd.12.trial090.stderr:2026-02-25T15:01:25.439+0000 787d4c0 -1 osd.12 0 log_to_monitors true 2026-02-25T15:01:25.456 INFO:tasks.ceph.osd.14.trial094.stderr:2026-02-25T15:01:25.453+0000 787d4c0 -1 osd.14 0 log_to_monitors true 2026-02-25T15:01:25.457 INFO:tasks.ceph.osd.11.trial189.stderr:2026-02-25T15:01:25.455+0000 787d4c0 -1 osd.11 0 log_to_monitors true 2026-02-25T15:01:25.461 INFO:tasks.ceph.osd.1.trial093.stderr:2026-02-25T15:01:25.457+0000 787d4c0 -1 osd.1 0 log_to_monitors true 2026-02-25T15:01:25.480 INFO:tasks.ceph.osd.6.trial094.stderr:2026-02-25T15:01:25.477+0000 787d4c0 -1 osd.6 0 log_to_monitors true 2026-02-25T15:01:25.535 INFO:tasks.ceph.osd.9.trial093.stderr:2026-02-25T15:01:25.531+0000 787d4c0 -1 osd.9 0 log_to_monitors true 2026-02-25T15:01:25.547 INFO:tasks.ceph.osd.15.trial189.stderr:2026-02-25T15:01:25.545+0000 787d4c0 -1 osd.15 0 log_to_monitors true 2026-02-25T15:01:25.563 INFO:tasks.ceph.osd.8.trial090.stderr:2026-02-25T15:01:25.559+0000 787d4c0 -1 osd.8 0 log_to_monitors true 2026-02-25T15:01:25.693 INFO:tasks.ceph.osd.7.trial189.stderr:2026-02-25T15:01:25.690+0000 787d4c0 -1 osd.7 0 log_to_monitors true 2026-02-25T15:01:25.694 INFO:tasks.ceph.osd.13.trial093.stderr:2026-02-25T15:01:25.689+0000 787d4c0 -1 osd.13 0 log_to_monitors true 2026-02-25T15:01:25.812 INFO:tasks.ceph.osd.3.trial189.stderr:2026-02-25T15:01:25.807+0000 787d4c0 -1 osd.3 0 log_to_monitors true 2026-02-25T15:01:27.438 INFO:tasks.ceph.osd.15.trial189.stderr:2026-02-25T15:01:27.437+0000 df77640 -1 osd.15 0 waiting for initial osdmap 2026-02-25T15:01:27.444 INFO:tasks.ceph.osd.2.trial094.stderr:2026-02-25T15:01:27.442+0000 df77640 -1 osd.2 0 waiting for initial osdmap 2026-02-25T15:01:27.454 INFO:tasks.ceph.osd.0.trial090.stderr:2026-02-25T15:01:27.451+0000 df77640 -1 osd.0 0 waiting for initial osdmap 2026-02-25T15:01:27.460 INFO:tasks.ceph.osd.9.trial093.stderr:2026-02-25T15:01:27.456+0000 cf64640 -1 osd.9 0 waiting for initial osdmap 2026-02-25T15:01:27.462 INFO:tasks.ceph.osd.5.trial093.stderr:2026-02-25T15:01:27.458+0000 cf64640 -1 osd.5 0 waiting for initial osdmap 2026-02-25T15:01:27.466 INFO:tasks.ceph.osd.10.trial094.stderr:2026-02-25T15:01:27.463+0000 df77640 -1 osd.10 0 waiting for initial osdmap 2026-02-25T15:01:27.477 INFO:tasks.ceph.osd.8.trial090.stderr:2026-02-25T15:01:27.472+0000 df77640 -1 osd.8 0 waiting for initial osdmap 2026-02-25T15:01:27.484 INFO:tasks.ceph.osd.11.trial189.stderr:2026-02-25T15:01:27.482+0000 df77640 -1 osd.11 0 waiting for initial osdmap 2026-02-25T15:01:27.484 INFO:tasks.ceph.osd.4.trial090.stderr:2026-02-25T15:01:27.477+0000 df77640 -1 osd.4 0 waiting for initial osdmap 2026-02-25T15:01:27.497 INFO:tasks.ceph.osd.12.trial090.stderr:2026-02-25T15:01:27.495+0000 cf64640 -1 osd.12 0 waiting for initial osdmap 2026-02-25T15:01:27.506 INFO:tasks.ceph.osd.14.trial094.stderr:2026-02-25T15:01:27.502+0000 df77640 -1 osd.14 0 waiting for initial osdmap 2026-02-25T15:01:27.516 INFO:tasks.ceph.osd.6.trial094.stderr:2026-02-25T15:01:27.513+0000 df77640 -1 osd.6 0 waiting for initial osdmap 2026-02-25T15:01:27.523 INFO:tasks.ceph.osd.1.trial093.stderr:2026-02-25T15:01:27.521+0000 cf64640 -1 osd.1 0 waiting for initial osdmap 2026-02-25T15:01:27.647 INFO:tasks.ceph.osd.15.trial189.stderr:2026-02-25T15:01:27.640+0000 1255f640 -1 osd.15 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:27.657 INFO:tasks.ceph.osd.2.trial094.stderr:2026-02-25T15:01:27.649+0000 1255f640 -1 osd.2 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:27.668 INFO:tasks.ceph.osd.9.trial093.stderr:2026-02-25T15:01:27.659+0000 1255f640 -1 osd.9 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:27.668 INFO:tasks.ceph.osd.5.trial093.stderr:2026-02-25T15:01:27.660+0000 16567640 -1 osd.5 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:27.674 INFO:tasks.ceph.osd.0.trial090.stderr:2026-02-25T15:01:27.666+0000 1255f640 -1 osd.0 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:27.677 INFO:tasks.ceph.osd.10.trial094.stderr:2026-02-25T15:01:27.670+0000 1255f640 -1 osd.10 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:27.688 INFO:tasks.ceph.osd.8.trial090.stderr:2026-02-25T15:01:27.680+0000 16567640 -1 osd.8 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:27.693 INFO:tasks.ceph.osd.11.trial189.stderr:2026-02-25T15:01:27.686+0000 1255f640 -1 osd.11 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:27.699 INFO:tasks.ceph.osd.4.trial090.stderr:2026-02-25T15:01:27.689+0000 1255f640 -1 osd.4 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:27.712 INFO:tasks.ceph.osd.14.trial094.stderr:2026-02-25T15:01:27.705+0000 1255f640 -1 osd.14 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:27.726 INFO:tasks.ceph.osd.6.trial094.stderr:2026-02-25T15:01:27.719+0000 1255f640 -1 osd.6 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:27.734 INFO:tasks.ceph.osd.1.trial093.stderr:2026-02-25T15:01:27.726+0000 16567640 -1 osd.1 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:27.737 INFO:tasks.ceph.osd.12.trial090.stderr:2026-02-25T15:01:27.729+0000 1255f640 -1 osd.12 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:28.490 DEBUG:teuthology.orchestra.run.trial090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-25T15:01:28.551 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.549+0000 7f0523d5c640 10 monclient: get_monmap_and_config 2026-02-25T15:01:28.551 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.549+0000 7f0523d5c640 10 monclient: build_initial_monmap 2026-02-25T15:01:28.551 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.549+0000 7f0523d5c640 10 monclient: monmap: 2026-02-25T15:01:28.552 INFO:teuthology.misc.health.trial090.stderr:epoch 0 2026-02-25T15:01:28.552 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:28.552 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:01:28.551304+0000 2026-02-25T15:01:28.552 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:01:28.551304+0000 2026-02-25T15:01:28.552 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:28.552 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:28.552 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:28.552 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:28.552 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:28.552 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:28.553 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.550+0000 7f0523d5c640 1 Processor -- start 2026-02-25T15:01:28.553 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.550+0000 7f0523d5c640 1 -- start start 2026-02-25T15:01:28.553 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.550+0000 7f0523d5c640 10 monclient: init 2026-02-25T15:01:28.553 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.550+0000 7f0523d5c640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:28.553 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.550+0000 7f0523d5c640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:28.553 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.550+0000 7f0523d5c640 10 monclient(hunting): picked mon.noname-b con 0x7f051c15ad50 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:28.553 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.550+0000 7f0523d5c640 10 monclient(hunting): picked mon.noname-a con 0x7f051c0b1730 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:28.553 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.550+0000 7f0523d5c640 10 monclient(hunting): picked mon.noname-c con 0x7f051c15b120 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:28.553 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.550+0000 7f0523d5c640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c0b4d50 con 0x7f051c0b1730 2026-02-25T15:01:28.553 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.550+0000 7f0523d5c640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c0b5030 con 0x7f051c15ad50 2026-02-25T15:01:28.553 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.550+0000 7f0523d5c640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c0b5310 con 0x7f051c15b120 2026-02-25T15:01:28.553 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.550+0000 7f0523d5c640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:28.553 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.550+0000 7f0523d5c640 10 monclient(hunting): authenticate will time out at 2978.151678s 2026-02-25T15:01:28.553 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.550+0000 7f0521ad1640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f051c15ad50 0x7f051c0b60d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:40144/0 (socket says 10.20.193.90:40144) 2026-02-25T15:01:28.553 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.550+0000 7f0521ad1640 1 -- 10.20.193.90:0/684396134 learned_addr learned my addr 10.20.193.90:0/684396134 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:28.554 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 1 -- 10.20.193.90:0/684396134 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 524968319 0 0) 0x7f051c0b5030 con 0x7f051c15ad50 2026-02-25T15:01:28.554 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 10 monclient(hunting): _negotiate 2026-02-25T15:01:28.554 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:28.554 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:28.554 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 10 monclient(hunting): my global_id is 4339 2026-02-25T15:01:28.554 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 1 -- 10.20.193.90:0/684396134 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0504003610 con 0x7f051c15ad50 2026-02-25T15:01:28.554 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 1 -- 10.20.193.90:0/684396134 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2045908915 0 0) 0x7f051c0b4d50 con 0x7f051c0b1730 2026-02-25T15:01:28.554 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 10 monclient(hunting): _negotiate 2026-02-25T15:01:28.555 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:28.555 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:28.555 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 10 monclient(hunting): my global_id is 4326 2026-02-25T15:01:28.555 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 1 -- 10.20.193.90:0/684396134 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f051c0b5030 con 0x7f051c0b1730 2026-02-25T15:01:28.555 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 1 -- 10.20.193.90:0/684396134 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3470626309 0 0) 0x7f051c0b5310 con 0x7f051c15b120 2026-02-25T15:01:28.555 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 10 monclient(hunting): _negotiate 2026-02-25T15:01:28.555 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:28.555 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:28.555 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 10 monclient(hunting): my global_id is 4343 2026-02-25T15:01:28.555 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.552+0000 7f0520acf640 1 -- 10.20.193.90:0/684396134 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f051c0b4d50 con 0x7f051c15b120 2026-02-25T15:01:28.556 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0520acf640 1 -- 10.20.193.90:0/684396134 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 418448132 0 0) 0x7f051c0b5030 con 0x7f051c0b1730 2026-02-25T15:01:28.556 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0520acf640 1 -- 10.20.193.90:0/684396134 >> v1:10.20.193.94:6789/0 conn(0x7f051c15b120 legacy=0x7f051c057640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:28.556 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0520acf640 1 -- 10.20.193.90:0/684396134 >> v1:10.20.193.93:6789/0 conn(0x7f051c15ad50 legacy=0x7f051c0b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:28.556 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0520acf640 10 monclient: _finish_hunting 0 2026-02-25T15:01:28.556 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0520acf640 1 monclient: found mon.noname-a 2026-02-25T15:01:28.556 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0520acf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:28.556 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0520acf640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:28.556 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0520acf640 1 -- 10.20.193.90:0/684396134 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c0b55f0 con 0x7f051c0b1730 2026-02-25T15:01:28.556 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0520acf640 10 monclient: _finish_auth 0 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0520acf640 10 monclient: _check_auth_tickets 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0520acf640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0520acf640 1 -- 10.20.193.90:0/684396134 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0510002df0 con 0x7f051c0b1730 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0520acf640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0520acf640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0520acf640 10 monclient: dump: 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0523d5c640 5 monclient: authenticate success, global_id 4326 2026-02-25T15:01:28.557 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.554+0000 7f0523d5c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:28.558 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0520acf640 1 -- 10.20.193.90:0/684396134 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0510002fc0 con 0x7f051c0b1730 2026-02-25T15:01:28.558 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0520acf640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:28.558 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0520acf640 1 -- 10.20.193.90:0/684396134 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0510003400 con 0x7f051c0b1730 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0520acf640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0520acf640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0520acf640 10 monclient: dump: 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0523d5c640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0523d5c640 4 set_mon_vals no callback set 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0523d5c640 10 monclient: shutdown 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0523d5c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0523d5c640 1 -- 10.20.193.90:0/684396134 >> v1:10.20.193.90:6789/0 conn(0x7f051c0b1730 legacy=0x7f051c05a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0523d5c640 1 -- 10.20.193.90:0/684396134 shutdown_connections 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0523d5c640 1 -- 10.20.193.90:0/684396134 >> 10.20.193.90:0/684396134 conn(0x7f051c0b3830 msgr2=0x7f051c0b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0523d5c640 1 -- 10.20.193.90:0/684396134 shutdown_connections 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0523d5c640 1 -- 10.20.193.90:0/684396134 wait complete. 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0523d5c640 10 monclient: build_initial_monmap 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.555+0000 7f0523d5c640 10 monclient: monmap: 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:epoch 0 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:28.559 INFO:teuthology.misc.health.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:created 0.000000 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 1 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 1 librados: starting msgr at 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 1 librados: starting objecter 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 1 Processor -- start 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 1 -- start start 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 1 librados: setting wanted keys 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 1 librados: calling monclient init 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 10 monclient: init 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 10 monclient(hunting): picked mon.noname-c con 0x7f051c0b1730 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 10 monclient(hunting): picked mon.noname-b con 0x7f051c15ad50 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 10 monclient(hunting): picked mon.noname-a con 0x7f051c15b120 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c1606b0 con 0x7f051c15b120 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c163bd0 con 0x7f051c15ad50 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c146650 con 0x7f051c0b1730 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f0523d5c640 10 monclient(hunting): authenticate will time out at 2978.157626s 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f05212d0640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f051c15ad50 0x7f051c160e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:40158/0 (socket says 10.20.193.90:40158) 2026-02-25T15:01:28.560 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.556+0000 7f05212d0640 1 -- 10.20.193.90:0/61088602 learned_addr learned my addr 10.20.193.90:0/61088602 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:28.561 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.557+0000 7f050a7fc640 10 client.?.objecter ms_handle_connect 0x7f051c15ad50 2026-02-25T15:01:28.561 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.557+0000 7f050a7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:28.561 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.557+0000 7f050a7fc640 10 client.?.objecter ms_handle_connect 0x7f051c0b1730 2026-02-25T15:01:28.561 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.557+0000 7f050a7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:28.561 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.557+0000 7f050a7fc640 10 client.?.objecter ms_handle_connect 0x7f051c15b120 2026-02-25T15:01:28.561 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.557+0000 7f050a7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:28.561 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2677722091 0 0) 0x7f051c163bd0 con 0x7f051c15ad50 2026-02-25T15:01:28.561 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:28.561 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:28.561 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:28.561 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 10 monclient(hunting): my global_id is 4342 2026-02-25T15:01:28.562 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04ec003610 con 0x7f051c15ad50 2026-02-25T15:01:28.562 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1913781240 0 0) 0x7f051c146650 con 0x7f051c0b1730 2026-02-25T15:01:28.562 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:28.562 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:28.562 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:28.562 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 10 monclient(hunting): my global_id is 4346 2026-02-25T15:01:28.562 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f051c163bd0 con 0x7f051c0b1730 2026-02-25T15:01:28.562 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1182903320 0 0) 0x7f051c1606b0 con 0x7f051c15b120 2026-02-25T15:01:28.562 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:28.562 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:28.562 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:28.562 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 10 monclient(hunting): my global_id is 4329 2026-02-25T15:01:28.562 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.558+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f051c146650 con 0x7f051c15b120 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.560+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 303615853 0 0) 0x7f04ec003610 con 0x7f051c15ad50 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.560+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f051c1606b0 con 0x7f051c15ad50 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.560+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0510003390 con 0x7f051c15ad50 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.560+0000 7f050a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.560+0000 7f050a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.560+0000 7f050a7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.560+0000 7f050a7fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.561+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1512345493 0 0) 0x7f051c163bd0 con 0x7f051c0b1730 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.561+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f04ec003610 con 0x7f051c0b1730 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.561+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f050c0030b0 con 0x7f051c0b1730 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.561+0000 7f050a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.561+0000 7f050a7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:28.563 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.561+0000 7f050a7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.561+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2603594128 0 0) 0x7f051c146650 con 0x7f051c15b120 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.561+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f051c163bd0 con 0x7f051c15b120 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.561+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0518003330 con 0x7f051c15b120 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.561+0000 7f050a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.561+0000 7f050a7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.561+0000 7f050a7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:28.564 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:28.565 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:28.565 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:28.565 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:28.565 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:28.565 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:28.565 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:28.565 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:28.565 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.562+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1899320144 0 0) 0x7f051c1606b0 con 0x7f051c15ad50 2026-02-25T15:01:28.565 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.562+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 >> v1:10.20.193.94:6789/0 conn(0x7f051c0b1730 legacy=0x7f051c1634c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:28.565 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.562+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 >> v1:10.20.193.90:6789/0 conn(0x7f051c15b120 legacy=0x7f051c149730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f050a7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f050a7fc640 1 monclient: found mon.b 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f050a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f050a7fc640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c146930 con 0x7f051c15ad50 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f050a7fc640 10 monclient: _finish_auth 0 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f050a7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f050a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f0523d5c640 5 monclient: authenticate success, global_id 4342 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f0523d5c640 10 monclient: _renew_subs 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f0523d5c640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f0523d5c640 1 -- 10.20.193.90:0/61088602 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f051c146c10 con 0x7f051c15ad50 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f0523d5c640 10 client.4342.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f0523d5c640 10 monclient: _renew_subs 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f0523d5c640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f0523d5c640 1 -- 10.20.193.90:0/61088602 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f051c161550 con 0x7f051c15ad50 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f0523d5c640 1 librados: init done 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f0523d5c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f0523d5c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f0523d5c640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:28.566 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.563+0000 7f0523d5c640 1 -- 10.20.193.90:0/61088602 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f051c161c10 con 0x7f051c15ad50 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.569+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0510002f80 con 0x7f051c15ad50 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.569+0000 7f050a7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.569+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0510003b00 con 0x7f051c15ad50 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.569+0000 7f050a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.569+0000 7f050bfff640 4 set_mon_vals no callback set 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.569+0000 7f050a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.569+0000 7f050a7fc640 10 monclient: dump: 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:28.571 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:28.573 INFO:tasks.ceph.osd.7.trial189.stderr:2026-02-25T15:01:28.570+0000 cf64640 -1 osd.7 0 waiting for initial osdmap 2026-02-25T15:01:28.573 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.571+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 4) ==== 103168+0+0 (unknown 1070002493 0 0) 0x7f0510003b00 con 0x7f051c15ad50 2026-02-25T15:01:28.574 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.572+0000 7f0521ad1640 10 monclient: get_auth_request con 0x7f04ec07a030 auth_method 0 2026-02-25T15:01:28.574 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.572+0000 7f050a7fc640 10 client.4342.objecter ms_handle_connect 0x7f04ec07a030 2026-02-25T15:01:28.575 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.573+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 6948+0+0 (unknown 1120946955 0 0) 0x7f0510095510 con 0x7f051c15ad50 2026-02-25T15:01:28.575 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.573+0000 7f050a7fc640 10 client.4342.objecter ms_dispatch2 0x7f051c000b90 osd_map(21..21 src has 1..21) 2026-02-25T15:01:28.575 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.573+0000 7f050a7fc640 3 client.4342.objecter handle_osd_map got epochs [21,21] > 0 2026-02-25T15:01:28.575 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.573+0000 7f050a7fc640 3 client.4342.objecter handle_osd_map decoding full epoch 21 2026-02-25T15:01:28.575 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.573+0000 7f050a7fc640 20 client.4342.objecter dump_active .. 0 homeless 2026-02-25T15:01:28.590 INFO:tasks.ceph.osd.3.trial189.stderr:2026-02-25T15:01:28.587+0000 cf64640 -1 osd.3 0 waiting for initial osdmap 2026-02-25T15:01:28.609 INFO:tasks.ceph.osd.13.trial093.stderr:2026-02-25T15:01:28.605+0000 cf64640 -1 osd.13 0 waiting for initial osdmap 2026-02-25T15:01:28.661 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.659+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f051005f280 con 0x7f051c15ad50 2026-02-25T15:01:28.661 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.659+0000 7f050a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:28.661 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.659+0000 7f050a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:28.778 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.776+0000 7f0523d5c640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:28.778 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.776+0000 7f0523d5c640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:28.778 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.776+0000 7f0523d5c640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:28.778 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.776+0000 7f0523d5c640 1 -- 10.20.193.90:0/61088602 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f051c159d00 con 0x7f051c15ad50 2026-02-25T15:01:28.787 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.785+0000 7f050a7fc640 1 -- 10.20.193.90:0/61088602 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+15942 (unknown 898900276 0 1890273630) 0x7f051005eb80 con 0x7f051c15ad50 2026-02-25T15:01:28.787 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.785+0000 7f050a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:28.787 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.785+0000 7f050a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:28.788 INFO:teuthology.misc.health.trial090.stdout: 2026-02-25T15:01:28.788 INFO:teuthology.misc.health.trial090.stdout:{"epoch":21,"fsid":"63c679b5-d219-4797-8adb-7340f3fd6819","created":"2026-02-25T15:00:56.256818+0000","modified":"2026-02-25T15:01:28.307914+0000","last_up_change":"2026-02-25T15:01:28.307914+0000","last_in_change":"2026-02-25T15:01:02.711189+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ad97006e-f5c3-4e36-97a0-7fa2b250344d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6808","nonce":3221464204}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6809","nonce":3221464204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6811","nonce":3221464204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6810","nonce":3221464204}]},"public_addr":"10.20.193.90:6808/3221464204","cluster_addr":"10.20.193.90:6809/3221464204","heartbeat_back_addr":"10.20.193.90:6811/3221464204","heartbeat_front_addr":"10.20.193.90:6810/3221464204","state":["exists","up"]},{"osd":1,"uuid":"25b02f30-96b9-43a3-9691-f06a1292e36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6804","nonce":3161035071}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6805","nonce":3161035071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6807","nonce":3161035071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6806","nonce":3161035071}]},"public_addr":"10.20.193.93:6804/3161035071","cluster_addr":"10.20.193.93:6805/3161035071","heartbeat_back_addr":"10.20.193.93:6807/3161035071","heartbeat_front_addr":"10.20.193.93:6806/3161035071","state":["exists","up"]},{"osd":2,"uuid":"aeed908b-d483-498b-83ba-632e51dd05e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6800","nonce":3705139403}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6801","nonce":3705139403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6803","nonce":3705139403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6802","nonce":3705139403}]},"public_addr":"10.20.193.94:6800/3705139403","cluster_addr":"10.20.193.94:6801/3705139403","heartbeat_back_addr":"10.20.193.94:6803/3705139403","heartbeat_front_addr":"10.20.193.94:6802/3705139403","state":["exists","up"]},{"osd":3,"uuid":"c7a30346-685f-4bd2-acaa-00b98ad3284c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d6c97301-55cc-4579-bc92-cb240c5ebe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6800","nonce":1446526134}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6802","nonce":1446526134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6805","nonce":1446526134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6803","nonce":1446526134}]},"public_addr":"10.20.193.90:6800/1446526134","cluster_addr":"10.20.193.90:6802/1446526134","heartbeat_back_addr":"10.20.193.90:6805/1446526134","heartbeat_front_addr":"10.20.193.90:6803/1446526134","state":["exists","up"]},{"osd":5,"uuid":"e3117058-2f0c-466b-a302-061fc4c120fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6800","nonce":887686167}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6801","nonce":887686167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6803","nonce":887686167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6802","nonce":887686167}]},"public_addr":"10.20.193.93:6800/887686167","cluster_addr":"10.20.193.93:6801/887686167","heartbeat_back_addr":"10.20.193.93:6803/887686167","heartbeat_front_addr":"10.20.193.93:6802/887686167","state":["exists","up"]},{"osd":6,"uuid":"f4b4a3ce-0a0b-4386-a433-dcbf65558ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6812","nonce":1114158993}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6813","nonce":1114158993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6815","nonce":1114158993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6814","nonce":1114158993}]},"public_addr":"10.20.193.94:6812/1114158993","cluster_addr":"10.20.193.94:6813/1114158993","heartbeat_back_addr":"10.20.193.94:6815/1114158993","heartbeat_front_addr":"10.20.193.94:6814/1114158993","state":["exists","up"]},{"osd":7,"uuid":"3337e549-6b6e-444e-997b-18a19beb5640","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"b292f7c4-9a41-4e48-876c-29281c5ec73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6801","nonce":2320036239}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6804","nonce":2320036239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6807","nonce":2320036239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6806","nonce":2320036239}]},"public_addr":"10.20.193.90:6801/2320036239","cluster_addr":"10.20.193.90:6804/2320036239","heartbeat_back_addr":"10.20.193.90:6807/2320036239","heartbeat_front_addr":"10.20.193.90:6806/2320036239","state":["exists","up"]},{"osd":9,"uuid":"954a7cbb-2f06-4cb9-ae68-380ac58a9f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6808","nonce":3039017972}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6809","nonce":3039017972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6811","nonce":3039017972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6810","nonce":3039017972}]},"public_addr":"10.20.193.93:6808/3039017972","cluster_addr":"10.20.193.93:6809/3039017972","heartbeat_back_addr":"10.20.193.93:6811/3039017972","heartbeat_front_addr":"10.20.193.93:6810/3039017972","state":["exists","up"]},{"osd":10,"uuid":"95ed1c74-8ca5-4317-9ef7-fc657b3cda70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6804","nonce":4185080112}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6805","nonce":4185080112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6807","nonce":4185080112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6806","nonce":4185080112}]},"public_addr":"10.20.193.94:6804/4185080112","cluster_addr":"10.20.193.94:6805/4185080112","heartbeat_back_addr":"10.20.193.94:6807/4185080112","heartbeat_front_addr":"10.20.193.94:6806/4185080112","state":["exists","up"]},{"osd":11,"uuid":"8f617b59-e8d1-42fb-8787-64e8c2117954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6801","nonce":4031575111}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6802","nonce":4031575111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6804","nonce":4031575111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6803","nonce":4031575111}]},"public_addr":"10.20.193.189:6801/4031575111","cluster_addr":"10.20.193.189:6802/4031575111","heartbeat_back_addr":"10.20.193.189:6804/4031575111","heartbeat_front_addr":"10.20.193.189:6803/4031575111","state":["exists","up"]},{"osd":12,"uuid":"4bad4371-2ca7-4023-8406-0275011eda9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6812","nonce":1139796254}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6813","nonce":1139796254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6815","nonce":1139796254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6814","nonce":1139796254}]},"public_addr":"10.20.193.90:6812/1139796254","cluster_addr":"10.20.193.90:6813/1139796254","heartbeat_back_addr":"10.20.193.90:6815/1139796254","heartbeat_front_addr":"10.20.193.90:6814/1139796254","state":["exists","up"]},{"osd":13,"uuid":"5d1ca952-93a5-43f3-9780-8bc85f9ed3d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"50853d20-2756-4720-b7aa-3020ded29189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6808","nonce":887271598}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6809","nonce":887271598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6811","nonce":887271598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6810","nonce":887271598}]},"public_addr":"10.20.193.94:6808/887271598","cluster_addr":"10.20.193.94:6809/887271598","heartbeat_back_addr":"10.20.193.94:6811/887271598","heartbeat_front_addr":"10.20.193.94:6810/887271598","state":["exists","up"]},{"osd":15,"uuid":"4445eda7-fb77-4f2c-8fda-77667df22b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6805","nonce":4039858964}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6806","nonce":4039858964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6808","nonce":4039858964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6807","nonce":4039858964}]},"public_addr":"10.20.193.189:6805/4039858964","cluster_addr":"10.20.193.189:6806/4039858964","heartbeat_back_addr":"10.20.193.189:6808/4039858964","heartbeat_front_addr":"10.20.193.189:6807/4039858964","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T15:01:28.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.787+0000 7f0523d5c640 10 librados: watch_flush enter 2026-02-25T15:01:28.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.787+0000 7f0523d5c640 10 librados: watch_flush exit 2026-02-25T15:01:28.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.787+0000 7f0523d5c640 20 client.4342.objecter shutdown clearing up homeless session... 2026-02-25T15:01:28.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.787+0000 7f0523d5c640 10 client.4342.objecter successfully canceled tick 2026-02-25T15:01:28.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.787+0000 7f0523d5c640 1 -- 10.20.193.90:0/61088602 >> v1:10.20.193.189:6800/3647375982 conn(0x7f04ec07a030 legacy=0x7f04ec09a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:28.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.787+0000 7f0523d5c640 10 monclient: shutdown 2026-02-25T15:01:28.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.787+0000 7f0523d5c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:28.791 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.787+0000 7f0523d5c640 1 -- 10.20.193.90:0/61088602 >> v1:10.20.193.93:6789/0 conn(0x7f051c15ad50 legacy=0x7f051c160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:28.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.787+0000 7f0523d5c640 1 -- 10.20.193.90:0/61088602 shutdown_connections 2026-02-25T15:01:28.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.787+0000 7f0523d5c640 1 -- 10.20.193.90:0/61088602 >> 10.20.193.90:0/61088602 conn(0x7f051c0b3830 msgr2=0x7f051c158a70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:28.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.788+0000 7f0523d5c640 1 -- 10.20.193.90:0/61088602 shutdown_connections 2026-02-25T15:01:28.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.788+0000 7f0523d5c640 1 -- 10.20.193.90:0/61088602 wait complete. 2026-02-25T15:01:28.792 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:28.788+0000 7f0523d5c640 1 librados: shutdown 2026-02-25T15:01:28.799 DEBUG:teuthology.misc:13 of 16 OSDs are up 2026-02-25T15:01:28.810 INFO:tasks.ceph.osd.7.trial189.stderr:2026-02-25T15:01:28.804+0000 16567640 -1 osd.7 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:28.827 INFO:tasks.ceph.osd.3.trial189.stderr:2026-02-25T15:01:28.820+0000 1255f640 -1 osd.3 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:28.858 INFO:tasks.ceph.osd.13.trial093.stderr:2026-02-25T15:01:28.850+0000 16967640 -1 osd.13 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T15:01:36.408 DEBUG:teuthology.orchestra.run.trial090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.466+0000 7f06b93b3640 10 monclient: get_monmap_and_config 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.466+0000 7f06b93b3640 10 monclient: build_initial_monmap 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.466+0000 7f06b93b3640 10 monclient: monmap: 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr:epoch 0 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:01:36.466966+0000 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:01:36.466966+0000 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.466+0000 7f06b93b3640 1 Processor -- start 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.466+0000 7f06b93b3640 1 -- start start 2026-02-25T15:01:36.467 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.466+0000 7f06b93b3640 10 monclient: init 2026-02-25T15:01:36.468 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.467+0000 7f06b93b3640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:36.468 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.467+0000 7f06b93b3640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:36.468 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.467+0000 7f06b93b3640 10 monclient(hunting): picked mon.noname-b con 0x7f06b4058da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:36.468 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.467+0000 7f06b93b3640 10 monclient(hunting): picked mon.noname-a con 0x7f06b40b0410 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:36.468 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.467+0000 7f06b93b3640 10 monclient(hunting): picked mon.noname-c con 0x7f06b41626a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:36.468 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.467+0000 7f06b93b3640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b41655c0 con 0x7f06b40b0410 2026-02-25T15:01:36.468 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.467+0000 7f06b93b3640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b40b12b0 con 0x7f06b4058da0 2026-02-25T15:01:36.468 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.467+0000 7f06b93b3640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b40b1590 con 0x7f06b41626a0 2026-02-25T15:01:36.468 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.467+0000 7f06b93b3640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:36.468 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.467+0000 7f06b93b3640 10 monclient(hunting): authenticate will time out at 2986.067431s 2026-02-25T15:01:36.468 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.467+0000 7f06b2ffd640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f06b4058da0 0x7f06b41647a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46194/0 (socket says 10.20.193.90:46194) 2026-02-25T15:01:36.468 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.467+0000 7f06b2ffd640 1 -- 10.20.193.90:0/3311694932 learned_addr learned my addr 10.20.193.90:0/3311694932 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:36.468 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.467+0000 7f06b27fc640 1 --1- 10.20.193.90:0/3311694932 >> v1:10.20.193.90:6789/0 conn(0x7f06b40b0410 0x7f06b4164eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44570/0 (socket says 10.20.193.90:44570) 2026-02-25T15:01:36.470 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.469+0000 7f06b1ffb640 1 -- 10.20.193.90:0/3311694932 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2202130398 0 0) 0x7f06b41655c0 con 0x7f06b40b0410 2026-02-25T15:01:36.470 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.469+0000 7f06b1ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:36.470 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.469+0000 7f06b1ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:36.470 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.469+0000 7f06b1ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:36.470 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.469+0000 7f06b1ffb640 10 monclient(hunting): my global_id is 4341 2026-02-25T15:01:36.470 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.469+0000 7f06b1ffb640 1 -- 10.20.193.90:0/3311694932 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06a0003610 con 0x7f06b40b0410 2026-02-25T15:01:36.470 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.469+0000 7f06b1ffb640 1 -- 10.20.193.90:0/3311694932 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2356980470 0 0) 0x7f06b40b12b0 con 0x7f06b4058da0 2026-02-25T15:01:36.470 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.469+0000 7f06b1ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:36.470 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.469+0000 7f06b1ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:36.470 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.469+0000 7f06b1ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:36.470 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.469+0000 7f06b1ffb640 10 monclient(hunting): my global_id is 4354 2026-02-25T15:01:36.470 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.469+0000 7f06b1ffb640 1 -- 10.20.193.90:0/3311694932 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06b41655c0 con 0x7f06b4058da0 2026-02-25T15:01:36.471 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.470+0000 7f06b1ffb640 1 -- 10.20.193.90:0/3311694932 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 763972511 0 0) 0x7f06b41655c0 con 0x7f06b4058da0 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.470+0000 7f06b1ffb640 1 -- 10.20.193.90:0/3311694932 >> v1:10.20.193.94:6789/0 conn(0x7f06b41626a0 legacy=0x7f06b41c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.470+0000 7f06b1ffb640 1 -- 10.20.193.90:0/3311694932 >> v1:10.20.193.90:6789/0 conn(0x7f06b40b0410 legacy=0x7f06b4164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.470+0000 7f06b1ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.470+0000 7f06b1ffb640 1 monclient: found mon.noname-b 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.470+0000 7f06b1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.470+0000 7f06b1ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.470+0000 7f06b1ffb640 1 -- 10.20.193.90:0/3311694932 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b40b1870 con 0x7f06b4058da0 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.470+0000 7f06b1ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.470+0000 7f06b1ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.470+0000 7f06b1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.471+0000 7f06b1ffb640 1 -- 10.20.193.90:0/3311694932 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f06a8002df0 con 0x7f06b4058da0 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.471+0000 7f06b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.471+0000 7f06b1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.471+0000 7f06b1ffb640 10 monclient: dump: 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.471+0000 7f06b93b3640 5 monclient: authenticate success, global_id 4354 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.471+0000 7f06b93b3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.471+0000 7f06b1ffb640 1 -- 10.20.193.90:0/3311694932 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f06a8003000 con 0x7f06b4058da0 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.471+0000 7f06b1ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.471+0000 7f06b1ffb640 1 -- 10.20.193.90:0/3311694932 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f06a8003440 con 0x7f06b4058da0 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.471+0000 7f06b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.471+0000 7f06b1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:36.472 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.471+0000 7f06b1ffb640 10 monclient: dump: 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.471+0000 7f06b93b3640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.471+0000 7f06b93b3640 4 set_mon_vals no callback set 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.472+0000 7f06b93b3640 10 monclient: shutdown 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.472+0000 7f06b93b3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.472+0000 7f06b93b3640 1 -- 10.20.193.90:0/3311694932 >> v1:10.20.193.93:6789/0 conn(0x7f06b4058da0 legacy=0x7f06b41647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.472+0000 7f06b93b3640 1 -- 10.20.193.90:0/3311694932 shutdown_connections 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.472+0000 7f06b93b3640 1 -- 10.20.193.90:0/3311694932 >> 10.20.193.90:0/3311694932 conn(0x7f06b40b65f0 msgr2=0x7f06b40b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.472+0000 7f06b93b3640 1 -- 10.20.193.90:0/3311694932 shutdown_connections 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.472+0000 7f06b93b3640 1 -- 10.20.193.90:0/3311694932 wait complete. 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.472+0000 7f06b93b3640 10 monclient: build_initial_monmap 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.472+0000 7f06b93b3640 10 monclient: monmap: 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:epoch 0 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:created 0.000000 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:36.473 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 1 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 1 librados: starting msgr at 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 1 librados: starting objecter 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 1 Processor -- start 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 1 -- start start 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 1 librados: setting wanted keys 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 1 librados: calling monclient init 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 10 monclient: init 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 10 monclient(hunting): picked mon.noname-b con 0x7f06b4058da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 10 monclient(hunting): picked mon.noname-c con 0x7f06b40b0410 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 10 monclient(hunting): picked mon.noname-a con 0x7f06b41626a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b41fe6e0 con 0x7f06b41626a0 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b41fe9c0 con 0x7f06b4058da0 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b41fa980 con 0x7f06b40b0410 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:36.474 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.473+0000 7f06b93b3640 10 monclient(hunting): authenticate will time out at 2986.073799s 2026-02-25T15:01:36.475 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.474+0000 7f06b2ffd640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f06b4058da0 0x7f06b41fde40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46200/0 (socket says 10.20.193.90:46200) 2026-02-25T15:01:36.475 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.474+0000 7f06b2ffd640 1 -- 10.20.193.90:0/3279302340 learned_addr learned my addr 10.20.193.90:0/3279302340 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:36.475 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.474+0000 7f06937fe640 10 client.?.objecter ms_handle_connect 0x7f06b41626a0 2026-02-25T15:01:36.475 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.474+0000 7f06937fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:36.475 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.474+0000 7f06937fe640 10 client.?.objecter ms_handle_connect 0x7f06b4058da0 2026-02-25T15:01:36.475 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.474+0000 7f06937fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:36.475 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.474+0000 7f06937fe640 10 client.?.objecter ms_handle_connect 0x7f06b40b0410 2026-02-25T15:01:36.475 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.474+0000 7f06937fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:36.476 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.475+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1127993012 0 0) 0x7f06b41fe6e0 con 0x7f06b41626a0 2026-02-25T15:01:36.476 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.475+0000 7f06937fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:36.476 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.475+0000 7f06937fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:36.476 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.475+0000 7f06937fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:36.476 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.475+0000 7f06937fe640 10 monclient(hunting): my global_id is 4344 2026-02-25T15:01:36.476 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.475+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0680003610 con 0x7f06b41626a0 2026-02-25T15:01:36.476 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.475+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4237418369 0 0) 0x7f06b41fe9c0 con 0x7f06b4058da0 2026-02-25T15:01:36.476 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.475+0000 7f06937fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:36.476 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.475+0000 7f06937fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:36.476 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.475+0000 7f06937fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:36.476 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.475+0000 7f06937fe640 10 monclient(hunting): my global_id is 4357 2026-02-25T15:01:36.476 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.475+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06b41fe6e0 con 0x7f06b4058da0 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.478+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3288827809 0 0) 0x7f06b41fe6e0 con 0x7f06b4058da0 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.478+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f06b41fe9c0 con 0x7f06b4058da0 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.478+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f06a8003350 con 0x7f06b4058da0 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.478+0000 7f06937fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.478+0000 7f06937fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.478+0000 7f06937fe640 10 monclient(hunting): dump: 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.478+0000 7f06937fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.478+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 160580450 0 0) 0x7f0680003610 con 0x7f06b41626a0 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.478+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f06b41fe6e0 con 0x7f06b41626a0 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.478+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f06a4003270 con 0x7f06b41626a0 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.478+0000 7f06937fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.478+0000 7f06937fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.478+0000 7f06937fe640 10 monclient(hunting): dump: 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:36.479 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.479+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1046292648 0 0) 0x7f06b41fe6e0 con 0x7f06b41626a0 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.479+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 >> v1:10.20.193.94:6789/0 conn(0x7f06b40b0410 legacy=0x7f06b4139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.479+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 >> v1:10.20.193.93:6789/0 conn(0x7f06b4058da0 legacy=0x7f06b41fde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.479+0000 7f06937fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.479+0000 7f06937fe640 1 monclient: found mon.a 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.479+0000 7f06937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.479+0000 7f06937fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.479+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b41fac60 con 0x7f06b41626a0 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.479+0000 7f06937fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.479+0000 7f06937fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.479+0000 7f06937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.479+0000 7f06b93b3640 5 monclient: authenticate success, global_id 4344 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.479+0000 7f06b93b3640 10 monclient: _renew_subs 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.479+0000 7f06b93b3640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.479+0000 7f06b93b3640 1 -- 10.20.193.90:0/3279302340 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f06b41faf40 con 0x7f06b41626a0 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.480+0000 7f06b93b3640 10 client.4344.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.480+0000 7f06b93b3640 10 monclient: _renew_subs 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.480+0000 7f06b93b3640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.480+0000 7f06b93b3640 1 -- 10.20.193.90:0/3279302340 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f06b41feca0 con 0x7f06b41626a0 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.480+0000 7f06b93b3640 1 librados: init done 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.480+0000 7f06b93b3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.480+0000 7f06b93b3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.480+0000 7f06b93b3640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:36.481 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.480+0000 7f06b93b3640 1 -- 10.20.193.90:0/3279302340 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b413a540 con 0x7f06b41626a0 2026-02-25T15:01:36.572 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.571+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f06a40036c0 con 0x7f06b41626a0 2026-02-25T15:01:36.572 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.571+0000 7f06937fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:36.572 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.571+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f06a4003a90 con 0x7f06b41626a0 2026-02-25T15:01:36.572 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.571+0000 7f06937fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:36.572 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.571+0000 7f06937fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:36.572 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.571+0000 7f06b17fa640 4 set_mon_vals no callback set 2026-02-25T15:01:36.572 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.571+0000 7f06937fe640 10 monclient: dump: 2026-02-25T15:01:36.572 INFO:teuthology.misc.health.trial090.stderr:epoch 1 2026-02-25T15:01:36.573 INFO:teuthology.misc.health.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:36.573 INFO:teuthology.misc.health.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.573 INFO:teuthology.misc.health.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.573 INFO:teuthology.misc.health.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:36.573 INFO:teuthology.misc.health.trial090.stderr:election_strategy: 3 2026-02-25T15:01:36.573 INFO:teuthology.misc.health.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:36.573 INFO:teuthology.misc.health.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:36.573 INFO:teuthology.misc.health.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:36.573 INFO:teuthology.misc.health.trial090.stderr: 2026-02-25T15:01:36.573 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.572+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f06a4003c30 con 0x7f06b41626a0 2026-02-25T15:01:36.573 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.573+0000 7f06b2ffd640 10 monclient: get_auth_request con 0x7f0680079f30 auth_method 0 2026-02-25T15:01:36.574 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.573+0000 7f06937fe640 10 client.4344.objecter ms_handle_connect 0x7f0680079f30 2026-02-25T15:01:36.576 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.575+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(24..24 src has 1..24) ==== 7757+0+0 (unknown 2066895719 0 0) 0x7f06a4094be0 con 0x7f06b41626a0 2026-02-25T15:01:36.576 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.575+0000 7f06937fe640 10 client.4344.objecter ms_dispatch2 0x7f06b4000b90 osd_map(24..24 src has 1..24) 2026-02-25T15:01:36.576 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.575+0000 7f06937fe640 3 client.4344.objecter handle_osd_map got epochs [24,24] > 0 2026-02-25T15:01:36.576 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.575+0000 7f06937fe640 3 client.4344.objecter handle_osd_map decoding full epoch 24 2026-02-25T15:01:36.576 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.575+0000 7f06937fe640 20 client.4344.objecter dump_active .. 0 homeless 2026-02-25T15:01:36.576 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.575+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f06a405e540 con 0x7f06b41626a0 2026-02-25T15:01:36.576 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.575+0000 7f06937fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:36.576 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.575+0000 7f06937fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:36.681 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.680+0000 7f06b93b3640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:36.681 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.680+0000 7f06b93b3640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:36.681 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.680+0000 7f06b93b3640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:36.681 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.680+0000 7f06b93b3640 1 -- 10.20.193.90:0/3279302340 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06b40af8b0 con 0x7f06b41626a0 2026-02-25T15:01:36.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.730+0000 7f06937fe640 1 -- 10.20.193.90:0/3279302340 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18917 (unknown 3496218867 0 3672995297) 0x7f06a405de40 con 0x7f06b41626a0 2026-02-25T15:01:36.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.730+0000 7f06937fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:36.731 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.730+0000 7f06937fe640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:36.732 INFO:teuthology.misc.health.trial090.stdout: 2026-02-25T15:01:36.732 INFO:teuthology.misc.health.trial090.stdout:{"epoch":24,"fsid":"63c679b5-d219-4797-8adb-7340f3fd6819","created":"2026-02-25T15:00:56.256818+0000","modified":"2026-02-25T15:01:31.574916+0000","last_up_change":"2026-02-25T15:01:29.405841+0000","last_in_change":"2026-02-25T15:01:02.711189+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T15:01:29.228894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ad97006e-f5c3-4e36-97a0-7fa2b250344d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6808","nonce":3221464204}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6809","nonce":3221464204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6811","nonce":3221464204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6810","nonce":3221464204}]},"public_addr":"10.20.193.90:6808/3221464204","cluster_addr":"10.20.193.90:6809/3221464204","heartbeat_back_addr":"10.20.193.90:6811/3221464204","heartbeat_front_addr":"10.20.193.90:6810/3221464204","state":["exists","up"]},{"osd":1,"uuid":"25b02f30-96b9-43a3-9691-f06a1292e36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6804","nonce":3161035071}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6805","nonce":3161035071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6807","nonce":3161035071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6806","nonce":3161035071}]},"public_addr":"10.20.193.93:6804/3161035071","cluster_addr":"10.20.193.93:6805/3161035071","heartbeat_back_addr":"10.20.193.93:6807/3161035071","heartbeat_front_addr":"10.20.193.93:6806/3161035071","state":["exists","up"]},{"osd":2,"uuid":"aeed908b-d483-498b-83ba-632e51dd05e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6800","nonce":3705139403}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6801","nonce":3705139403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6803","nonce":3705139403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6802","nonce":3705139403}]},"public_addr":"10.20.193.94:6800/3705139403","cluster_addr":"10.20.193.94:6801/3705139403","heartbeat_back_addr":"10.20.193.94:6803/3705139403","heartbeat_front_addr":"10.20.193.94:6802/3705139403","state":["exists","up"]},{"osd":3,"uuid":"c7a30346-685f-4bd2-acaa-00b98ad3284c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6813","nonce":2702305154}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6814","nonce":2702305154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6816","nonce":2702305154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6815","nonce":2702305154}]},"public_addr":"10.20.193.189:6813/2702305154","cluster_addr":"10.20.193.189:6814/2702305154","heartbeat_back_addr":"10.20.193.189:6816/2702305154","heartbeat_front_addr":"10.20.193.189:6815/2702305154","state":["exists","up"]},{"osd":4,"uuid":"d6c97301-55cc-4579-bc92-cb240c5ebe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6800","nonce":1446526134}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6802","nonce":1446526134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6805","nonce":1446526134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6803","nonce":1446526134}]},"public_addr":"10.20.193.90:6800/1446526134","cluster_addr":"10.20.193.90:6802/1446526134","heartbeat_back_addr":"10.20.193.90:6805/1446526134","heartbeat_front_addr":"10.20.193.90:6803/1446526134","state":["exists","up"]},{"osd":5,"uuid":"e3117058-2f0c-466b-a302-061fc4c120fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6800","nonce":887686167}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6801","nonce":887686167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6803","nonce":887686167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6802","nonce":887686167}]},"public_addr":"10.20.193.93:6800/887686167","cluster_addr":"10.20.193.93:6801/887686167","heartbeat_back_addr":"10.20.193.93:6803/887686167","heartbeat_front_addr":"10.20.193.93:6802/887686167","state":["exists","up"]},{"osd":6,"uuid":"f4b4a3ce-0a0b-4386-a433-dcbf65558ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6812","nonce":1114158993}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6813","nonce":1114158993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6815","nonce":1114158993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6814","nonce":1114158993}]},"public_addr":"10.20.193.94:6812/1114158993","cluster_addr":"10.20.193.94:6813/1114158993","heartbeat_back_addr":"10.20.193.94:6815/1114158993","heartbeat_front_addr":"10.20.193.94:6814/1114158993","state":["exists","up"]},{"osd":7,"uuid":"3337e549-6b6e-444e-997b-18a19beb5640","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6809","nonce":1708410075}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6810","nonce":1708410075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6812","nonce":1708410075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6811","nonce":1708410075}]},"public_addr":"10.20.193.189:6809/1708410075","cluster_addr":"10.20.193.189:6810/1708410075","heartbeat_back_addr":"10.20.193.189:6812/1708410075","heartbeat_front_addr":"10.20.193.189:6811/1708410075","state":["exists","up"]},{"osd":8,"uuid":"b292f7c4-9a41-4e48-876c-29281c5ec73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6801","nonce":2320036239}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6804","nonce":2320036239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6807","nonce":2320036239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6806","nonce":2320036239}]},"public_addr":"10.20.193.90:6801/2320036239","cluster_addr":"10.20.193.90:6804/2320036239","heartbeat_back_addr":"10.20.193.90:6807/2320036239","heartbeat_front_addr":"10.20.193.90:6806/2320036239","state":["exists","up"]},{"osd":9,"uuid":"954a7cbb-2f06-4cb9-ae68-380ac58a9f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6808","nonce":3039017972}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6809","nonce":3039017972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6811","nonce":3039017972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6810","nonce":3039017972}]},"public_addr":"10.20.193.93:6808/3039017972","cluster_addr":"10.20.193.93:6809/3039017972","heartbeat_back_addr":"10.20.193.93:6811/3039017972","heartbeat_front_addr":"10.20.193.93:6810/3039017972","state":["exists","up"]},{"osd":10,"uuid":"95ed1c74-8ca5-4317-9ef7-fc657b3cda70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6804","nonce":4185080112}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6805","nonce":4185080112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6807","nonce":4185080112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6806","nonce":4185080112}]},"public_addr":"10.20.193.94:6804/4185080112","cluster_addr":"10.20.193.94:6805/4185080112","heartbeat_back_addr":"10.20.193.94:6807/4185080112","heartbeat_front_addr":"10.20.193.94:6806/4185080112","state":["exists","up"]},{"osd":11,"uuid":"8f617b59-e8d1-42fb-8787-64e8c2117954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6801","nonce":4031575111}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6802","nonce":4031575111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6804","nonce":4031575111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6803","nonce":4031575111}]},"public_addr":"10.20.193.189:6801/4031575111","cluster_addr":"10.20.193.189:6802/4031575111","heartbeat_back_addr":"10.20.193.189:6804/4031575111","heartbeat_front_addr":"10.20.193.189:6803/4031575111","state":["exists","up"]},{"osd":12,"uuid":"4bad4371-2ca7-4023-8406-0275011eda9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6812","nonce":1139796254}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6813","nonce":1139796254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6815","nonce":1139796254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6814","nonce":1139796254}]},"public_addr":"10.20.193.90:6812/1139796254","cluster_addr":"10.20.193.90:6813/1139796254","heartbeat_back_addr":"10.20.193.90:6815/1139796254","heartbeat_front_addr":"10.20.193.90:6814/1139796254","state":["exists","up"]},{"osd":13,"uuid":"5d1ca952-93a5-43f3-9780-8bc85f9ed3d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6812","nonce":1850919728}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6813","nonce":1850919728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6815","nonce":1850919728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6814","nonce":1850919728}]},"public_addr":"10.20.193.93:6812/1850919728","cluster_addr":"10.20.193.93:6813/1850919728","heartbeat_back_addr":"10.20.193.93:6815/1850919728","heartbeat_front_addr":"10.20.193.93:6814/1850919728","state":["exists","up"]},{"osd":14,"uuid":"50853d20-2756-4720-b7aa-3020ded29189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6808","nonce":887271598}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6809","nonce":887271598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6811","nonce":887271598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6810","nonce":887271598}]},"public_addr":"10.20.193.94:6808/887271598","cluster_addr":"10.20.193.94:6809/887271598","heartbeat_back_addr":"10.20.193.94:6811/887271598","heartbeat_front_addr":"10.20.193.94:6810/887271598","state":["exists","up"]},{"osd":15,"uuid":"4445eda7-fb77-4f2c-8fda-77667df22b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6805","nonce":4039858964}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6806","nonce":4039858964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6808","nonce":4039858964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6807","nonce":4039858964}]},"public_addr":"10.20.193.189:6805/4039858964","cluster_addr":"10.20.193.189:6806/4039858964","heartbeat_back_addr":"10.20.193.189:6808/4039858964","heartbeat_front_addr":"10.20.193.189:6807/4039858964","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.787596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.833159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.652364+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:27.190420+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.727824+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.707073+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.887715+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:27.132243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.999569+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.929271+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.761416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.870660+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.873627+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:27.124178+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.868032+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.940190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T15:01:36.734 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.732+0000 7f06b93b3640 10 librados: watch_flush enter 2026-02-25T15:01:36.734 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.732+0000 7f06b93b3640 10 librados: watch_flush exit 2026-02-25T15:01:36.734 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.732+0000 7f06b93b3640 20 client.4344.objecter shutdown clearing up homeless session... 2026-02-25T15:01:36.734 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.732+0000 7f06b93b3640 10 client.4344.objecter successfully canceled tick 2026-02-25T15:01:36.734 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.732+0000 7f06b93b3640 1 -- 10.20.193.90:0/3279302340 >> v1:10.20.193.189:6800/3647375982 conn(0x7f0680079f30 legacy=0x7f068009a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:36.735 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.732+0000 7f06b93b3640 10 monclient: shutdown 2026-02-25T15:01:36.735 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.732+0000 7f06b93b3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:36.735 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.732+0000 7f06b93b3640 1 -- 10.20.193.90:0/3279302340 >> v1:10.20.193.90:6789/0 conn(0x7f06b41626a0 legacy=0x7f06b40a7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:36.735 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.732+0000 7f06b93b3640 1 -- 10.20.193.90:0/3279302340 shutdown_connections 2026-02-25T15:01:36.735 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.732+0000 7f06b93b3640 1 -- 10.20.193.90:0/3279302340 >> 10.20.193.90:0/3279302340 conn(0x7f06b40b65f0 msgr2=0x7f06b4059bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:36.735 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.732+0000 7f06b93b3640 1 -- 10.20.193.90:0/3279302340 shutdown_connections 2026-02-25T15:01:36.735 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.732+0000 7f06b93b3640 1 -- 10.20.193.90:0/3279302340 wait complete. 2026-02-25T15:01:36.735 INFO:teuthology.misc.health.trial090.stderr:2026-02-25T15:01:36.732+0000 7f06b93b3640 1 librados: shutdown 2026-02-25T15:01:36.742 DEBUG:teuthology.misc:16 of 16 OSDs are up 2026-02-25T15:01:36.742 INFO:tasks.ceph:Creating RBD pool 2026-02-25T15:01:36.742 DEBUG:teuthology.orchestra.run.trial090:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-25T15:01:36.812 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.811+0000 7f1ca55aa640 10 monclient: get_monmap_and_config 2026-02-25T15:01:36.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.811+0000 7f1ca55aa640 10 monclient: build_initial_monmap 2026-02-25T15:01:36.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.811+0000 7f1ca55aa640 10 monclient: monmap: 2026-02-25T15:01:36.813 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:36.813 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:36.813 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:36.811807+0000 2026-02-25T15:01:36.813 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:36.811807+0000 2026-02-25T15:01:36.813 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:36.813 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:36.813 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:36.813 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:36.813 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:36.814 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:36.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.811+0000 7f1ca55aa640 1 Processor -- start 2026-02-25T15:01:36.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.811+0000 7f1ca55aa640 1 -- start start 2026-02-25T15:01:36.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.811+0000 7f1ca55aa640 10 monclient: init 2026-02-25T15:01:36.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.812+0000 7f1ca55aa640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:36.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.812+0000 7f1ca55aa640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:36.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.812+0000 7f1ca55aa640 10 monclient(hunting): picked mon.noname-a con 0x7f1ca015afd0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:36.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.812+0000 7f1ca55aa640 10 monclient(hunting): picked mon.noname-b con 0x7f1ca015b3a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:36.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.812+0000 7f1ca55aa640 10 monclient(hunting): picked mon.noname-c con 0x7f1ca0163890 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:36.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.812+0000 7f1ca55aa640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca0162380 con 0x7f1ca015afd0 2026-02-25T15:01:36.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.812+0000 7f1ca55aa640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca00b1c30 con 0x7f1ca015b3a0 2026-02-25T15:01:36.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.812+0000 7f1ca55aa640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca00b1f10 con 0x7f1ca0163890 2026-02-25T15:01:36.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.812+0000 7f1ca55aa640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:36.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.812+0000 7f1ca55aa640 10 monclient(hunting): authenticate will time out at 2986.412472s 2026-02-25T15:01:36.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.812+0000 7f1c9effd640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f1ca015afd0 0x7f1ca0059be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44582/0 (socket says 10.20.193.90:44582) 2026-02-25T15:01:36.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.812+0000 7f1c9effd640 1 -- 10.20.193.90:0/2828866286 learned_addr learned my addr 10.20.193.90:0/2828866286 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:36.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.814+0000 7f1c9dffb640 1 -- 10.20.193.90:0/2828866286 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4009857822 0 0) 0x7f1ca00b1c30 con 0x7f1ca015b3a0 2026-02-25T15:01:36.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.814+0000 7f1c9dffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:36.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.814+0000 7f1c9dffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:36.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.814+0000 7f1c9dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:36.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.814+0000 7f1c9dffb640 10 monclient(hunting): my global_id is 4360 2026-02-25T15:01:36.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.814+0000 7f1c9dffb640 1 -- 10.20.193.90:0/2828866286 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c8c003610 con 0x7f1ca015b3a0 2026-02-25T15:01:36.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.814+0000 7f1c9dffb640 1 -- 10.20.193.90:0/2828866286 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4207027170 0 0) 0x7f1ca0162380 con 0x7f1ca015afd0 2026-02-25T15:01:36.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.814+0000 7f1c9dffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:36.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.814+0000 7f1c9dffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:36.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.814+0000 7f1c9dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:36.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.814+0000 7f1c9dffb640 10 monclient(hunting): my global_id is 4347 2026-02-25T15:01:36.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.814+0000 7f1c9dffb640 1 -- 10.20.193.90:0/2828866286 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ca00b1c30 con 0x7f1ca015afd0 2026-02-25T15:01:36.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1c9dffb640 1 -- 10.20.193.90:0/2828866286 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1988719575 0 0) 0x7f1ca00b1c30 con 0x7f1ca015afd0 2026-02-25T15:01:36.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1c9dffb640 1 -- 10.20.193.90:0/2828866286 >> v1:10.20.193.94:6789/0 conn(0x7f1ca0163890 legacy=0x7f1ca0164480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:36.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1c9dffb640 1 -- 10.20.193.90:0/2828866286 >> v1:10.20.193.93:6789/0 conn(0x7f1ca015b3a0 legacy=0x7f1ca005d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:36.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1c9dffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:36.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1c9dffb640 1 monclient: found mon.noname-a 2026-02-25T15:01:36.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1c9dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:36.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1c9dffb640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:36.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1c9dffb640 1 -- 10.20.193.90:0/2828866286 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca00b21f0 con 0x7f1ca015afd0 2026-02-25T15:01:36.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1c9dffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:36.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1c9dffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:36.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1c9dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:36.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1c9dffb640 1 -- 10.20.193.90:0/2828866286 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1c94002df0 con 0x7f1ca015afd0 2026-02-25T15:01:36.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1ca55aa640 5 monclient: authenticate success, global_id 4347 2026-02-25T15:01:36.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1ca55aa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:36.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1c9dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:36.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1c9dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:36.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1c9dffb640 10 monclient: dump: 2026-02-25T15:01:36.831 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:36.831 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:36.832 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.832 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.832 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:36.832 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:36.832 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:36.832 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:36.832 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:36.833 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:36.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.816+0000 7f1ca55aa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:36.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1c9dffb640 1 -- 10.20.193.90:0/2828866286 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1c94003000 con 0x7f1ca015afd0 2026-02-25T15:01:36.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1c9dffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:36.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1c9dffb640 1 -- 10.20.193.90:0/2828866286 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1c94003440 con 0x7f1ca015afd0 2026-02-25T15:01:36.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1ca55aa640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:36.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1ca55aa640 4 set_mon_vals no callback set 2026-02-25T15:01:36.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1c9dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:36.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1c9dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:36.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1c9dffb640 10 monclient: dump: 2026-02-25T15:01:36.834 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:36.834 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:36.834 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.835 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.835 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:36.835 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:36.835 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:36.835 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:36.835 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:36.835 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:36.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1ca55aa640 10 monclient: shutdown 2026-02-25T15:01:36.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1ca55aa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:36.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1ca55aa640 1 -- 10.20.193.90:0/2828866286 >> v1:10.20.193.90:6789/0 conn(0x7f1ca015afd0 legacy=0x7f1ca0059be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:36.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1ca55aa640 1 -- 10.20.193.90:0/2828866286 shutdown_connections 2026-02-25T15:01:36.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1ca55aa640 1 -- 10.20.193.90:0/2828866286 >> 10.20.193.90:0/2828866286 conn(0x7f1ca00b2f10 msgr2=0x7f1ca00b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:36.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1ca55aa640 1 -- 10.20.193.90:0/2828866286 shutdown_connections 2026-02-25T15:01:36.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1ca55aa640 1 -- 10.20.193.90:0/2828866286 wait complete. 2026-02-25T15:01:36.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1ca55aa640 10 monclient: build_initial_monmap 2026-02-25T15:01:36.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.817+0000 7f1ca55aa640 10 monclient: monmap: 2026-02-25T15:01:36.837 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:36.837 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:36.837 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:36.837 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:36.837 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:36.838 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:36.838 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:36.838 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:36.838 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:36.838 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:36.838 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 1 librados: starting msgr at 2026-02-25T15:01:36.838 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 1 librados: starting objecter 2026-02-25T15:01:36.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:36.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 1 Processor -- start 2026-02-25T15:01:36.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 1 -- start start 2026-02-25T15:01:36.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 1 librados: setting wanted keys 2026-02-25T15:01:36.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 1 librados: calling monclient init 2026-02-25T15:01:36.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 10 monclient: init 2026-02-25T15:01:36.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:36.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:36.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 10 monclient(hunting): picked mon.noname-c con 0x7f1ca015afd0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:36.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 10 monclient(hunting): picked mon.noname-a con 0x7f1ca015b3a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:36.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 10 monclient(hunting): picked mon.noname-b con 0x7f1ca0163890 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:36.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca013aa80 con 0x7f1ca015b3a0 2026-02-25T15:01:36.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca01307c0 con 0x7f1ca0163890 2026-02-25T15:01:36.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca0130aa0 con 0x7f1ca015afd0 2026-02-25T15:01:36.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:36.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1ca55aa640 10 monclient(hunting): authenticate will time out at 2986.418583s 2026-02-25T15:01:36.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1c9e7fc640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f1ca015b3a0 0x7f1ca0055d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44586/0 (socket says 10.20.193.90:44586) 2026-02-25T15:01:36.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1c9e7fc640 1 -- 10.20.193.90:0/2460194167 learned_addr learned my addr 10.20.193.90:0/2460194167 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:36.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.818+0000 7f1c7f7fe640 10 client.?.objecter ms_handle_connect 0x7f1ca015b3a0 2026-02-25T15:01:36.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.819+0000 7f1c7f7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:36.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.819+0000 7f1c7f7fe640 10 client.?.objecter ms_handle_connect 0x7f1ca015afd0 2026-02-25T15:01:36.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.819+0000 7f1c7f7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:36.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.819+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1037271287 0 0) 0x7f1ca013aa80 con 0x7f1ca015b3a0 2026-02-25T15:01:36.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.819+0000 7f1c7f7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:36.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.819+0000 7f1c7f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:36.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.819+0000 7f1c7f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:36.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.819+0000 7f1c7f7fe640 10 monclient(hunting): my global_id is 4350 2026-02-25T15:01:36.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.819+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c6c003610 con 0x7f1ca015b3a0 2026-02-25T15:01:36.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.820+0000 7f1c7f7fe640 10 client.?.objecter ms_handle_connect 0x7f1ca0163890 2026-02-25T15:01:36.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.820+0000 7f1c7f7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:36.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.821+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4202577744 0 0) 0x7f1ca01307c0 con 0x7f1ca0163890 2026-02-25T15:01:36.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.821+0000 7f1c7f7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:36.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.821+0000 7f1c7f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:36.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.821+0000 7f1c7f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:36.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.821+0000 7f1c7f7fe640 10 monclient(hunting): my global_id is 4363 2026-02-25T15:01:36.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.821+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ca013aa80 con 0x7f1ca0163890 2026-02-25T15:01:36.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.822+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1571455753 0 0) 0x7f1c6c003610 con 0x7f1ca015b3a0 2026-02-25T15:01:36.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.822+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1ca01307c0 con 0x7f1ca015b3a0 2026-02-25T15:01:36.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.822+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1c880032b0 con 0x7f1ca015b3a0 2026-02-25T15:01:36.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.822+0000 7f1c7f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:36.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.822+0000 7f1c7f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:36.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.822+0000 7f1c7f7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:36.845 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:36.845 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:36.845 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.845 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.845 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:36.845 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:36.845 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:36.845 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:36.846 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:36.846 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:36.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.822+0000 7f1c7f7fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:36.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 547339687 0 0) 0x7f1ca013aa80 con 0x7f1ca0163890 2026-02-25T15:01:36.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1c6c003610 con 0x7f1ca0163890 2026-02-25T15:01:36.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1c90003270 con 0x7f1ca0163890 2026-02-25T15:01:36.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:36.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:36.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:36.847 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:36.847 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:36.847 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.847 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.847 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:36.847 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:36.847 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:36.847 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:36.847 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:36.847 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:36.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 135267424 0 0) 0x7f1ca01307c0 con 0x7f1ca015b3a0 2026-02-25T15:01:36.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 >> v1:10.20.193.94:6789/0 conn(0x7f1ca015afd0 legacy=0x7f1ca013a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:36.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 >> v1:10.20.193.93:6789/0 conn(0x7f1ca0163890 legacy=0x7f1ca0158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:36.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:36.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 1 monclient: found mon.a 2026-02-25T15:01:36.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:36.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:36.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca0130d80 con 0x7f1ca015b3a0 2026-02-25T15:01:36.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:36.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:36.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1c7f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:36.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1ca55aa640 5 monclient: authenticate success, global_id 4350 2026-02-25T15:01:36.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1ca55aa640 10 monclient: _renew_subs 2026-02-25T15:01:36.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1ca55aa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:36.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.824+0000 7f1ca55aa640 1 -- 10.20.193.90:0/2460194167 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca0131060 con 0x7f1ca015b3a0 2026-02-25T15:01:36.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.825+0000 7f1ca55aa640 10 client.4350.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:36.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.825+0000 7f1ca55aa640 10 monclient: _renew_subs 2026-02-25T15:01:36.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.825+0000 7f1ca55aa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:36.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.825+0000 7f1ca55aa640 1 -- 10.20.193.90:0/2460194167 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f1ca0131620 con 0x7f1ca015b3a0 2026-02-25T15:01:36.850 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.825+0000 7f1ca55aa640 1 librados: init done 2026-02-25T15:01:36.850 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.825+0000 7f1ca55aa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:36.850 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.825+0000 7f1ca55aa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:36.850 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.825+0000 7f1ca55aa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:36.850 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.825+0000 7f1ca55aa640 1 -- 10.20.193.90:0/2460194167 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca00b17a0 con 0x7f1ca015b3a0 2026-02-25T15:01:36.920 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.918+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1c880034c0 con 0x7f1ca015b3a0 2026-02-25T15:01:36.921 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.918+0000 7f1c7f7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:36.921 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.918+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1c88003bd0 con 0x7f1ca015b3a0 2026-02-25T15:01:36.921 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.918+0000 7f1c7f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:36.921 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.918+0000 7f1c7f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:36.921 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.918+0000 7f1c7f7fe640 10 monclient: dump: 2026-02-25T15:01:36.921 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:36.921 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:36.922 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.922 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:36.922 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:36.922 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:36.922 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:36.922 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:36.922 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:36.923 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:36.923 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.918+0000 7f1c9d7fa640 4 set_mon_vals no callback set 2026-02-25T15:01:36.923 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.919+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f1c88003db0 con 0x7f1ca015b3a0 2026-02-25T15:01:36.925 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.919+0000 7f1c9effd640 10 monclient: get_auth_request con 0x7f1c6c079dd0 auth_method 0 2026-02-25T15:01:36.925 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.920+0000 7f1c7f7fe640 10 client.4350.objecter ms_handle_connect 0x7f1c6c079dd0 2026-02-25T15:01:36.925 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.922+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(24..24 src has 1..24) ==== 7757+0+0 (unknown 2066895719 0 0) 0x7f1c88095dd0 con 0x7f1ca015b3a0 2026-02-25T15:01:36.925 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.922+0000 7f1c7f7fe640 10 client.4350.objecter ms_dispatch2 0x7f1ca0000b90 osd_map(24..24 src has 1..24) 2026-02-25T15:01:36.925 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.922+0000 7f1c7f7fe640 3 client.4350.objecter handle_osd_map got epochs [24,24] > 0 2026-02-25T15:01:36.925 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.922+0000 7f1c7f7fe640 3 client.4350.objecter handle_osd_map decoding full epoch 24 2026-02-25T15:01:36.926 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.922+0000 7f1c7f7fe640 20 client.4350.objecter dump_active .. 0 homeless 2026-02-25T15:01:36.926 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.922+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f1c8805f730 con 0x7f1ca015b3a0 2026-02-25T15:01:36.926 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.922+0000 7f1c7f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:36.926 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:36.922+0000 7f1c7f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:37.031 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.029+0000 7f1ca55aa640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-25T15:01:37.031 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.029+0000 7f1ca55aa640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-25T15:01:37.031 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.029+0000 7f1ca55aa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:37.031 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.029+0000 7f1ca55aa640 1 -- 10.20.193.90:0/2460194167 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f1ca01647c0 con 0x7f1ca015b3a0 2026-02-25T15:01:37.375 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.374+0000 7f1c7f7fe640 1 -- 10.20.193.90:0/2460194167 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v25) ==== 109+0+0 (unknown 2655803957 0 0) 0x7f1c8805f030 con 0x7f1ca015b3a0 2026-02-25T15:01:37.375 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.374+0000 7f1c7f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-25T15:01:37.376 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.374+0000 7f1c7f7fe640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2026-02-25T15:01:37.376 INFO:teuthology.orchestra.run.trial090.stderr:pool 'rbd' created 2026-02-25T15:01:37.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.377+0000 7f1ca55aa640 10 librados: watch_flush enter 2026-02-25T15:01:37.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.377+0000 7f1ca55aa640 10 librados: watch_flush exit 2026-02-25T15:01:37.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.377+0000 7f1ca55aa640 20 client.4350.objecter shutdown clearing up homeless session... 2026-02-25T15:01:37.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.377+0000 7f1ca55aa640 10 client.4350.objecter successfully canceled tick 2026-02-25T15:01:37.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.377+0000 7f1ca55aa640 1 -- 10.20.193.90:0/2460194167 >> v1:10.20.193.189:6800/3647375982 conn(0x7f1c6c079dd0 legacy=0x7f1c6c09a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:37.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.377+0000 7f1ca55aa640 10 monclient: shutdown 2026-02-25T15:01:37.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.377+0000 7f1ca55aa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:37.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.377+0000 7f1ca55aa640 1 -- 10.20.193.90:0/2460194167 >> v1:10.20.193.90:6789/0 conn(0x7f1ca015b3a0 legacy=0x7f1ca0055d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:37.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.377+0000 7f1ca55aa640 1 -- 10.20.193.90:0/2460194167 shutdown_connections 2026-02-25T15:01:37.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.377+0000 7f1ca55aa640 1 -- 10.20.193.90:0/2460194167 >> 10.20.193.90:0/2460194167 conn(0x7f1ca00b2f10 msgr2=0x7f1ca005d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:37.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.377+0000 7f1ca55aa640 1 -- 10.20.193.90:0/2460194167 shutdown_connections 2026-02-25T15:01:37.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.377+0000 7f1ca55aa640 1 -- 10.20.193.90:0/2460194167 wait complete. 2026-02-25T15:01:37.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.377+0000 7f1ca55aa640 1 librados: shutdown 2026-02-25T15:01:37.387 DEBUG:teuthology.orchestra.run.trial090:> rbd --cluster ceph pool init rbd 2026-02-25T15:01:37.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.421+0000 7fb4e26a4300 10 monclient: get_monmap_and_config 2026-02-25T15:01:37.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.421+0000 7fb4e26a4300 10 monclient: build_initial_monmap 2026-02-25T15:01:37.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.421+0000 7fb4e26a4300 10 monclient: monmap: 2026-02-25T15:01:37.423 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:37.423 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:37.423 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:37.422527+0000 2026-02-25T15:01:37.424 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:37.422527+0000 2026-02-25T15:01:37.424 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:37.424 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:37.424 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:37.424 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:37.424 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:37.424 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:37.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.422+0000 7fb4e26a4300 1 Processor -- start 2026-02-25T15:01:37.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.422+0000 7fb4e26a4300 1 -- start start 2026-02-25T15:01:37.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.422+0000 7fb4e26a4300 10 monclient: init 2026-02-25T15:01:37.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.422+0000 7fb4e26a4300 10 monclient: _reopen_session rank -1 2026-02-25T15:01:37.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.423+0000 7fb4e26a4300 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:37.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.423+0000 7fb4e26a4300 10 monclient(hunting): picked mon.noname-c con 0x56420f0195d0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:37.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.423+0000 7fb4e26a4300 10 monclient(hunting): picked mon.noname-a con 0x56420f102dc0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:37.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.423+0000 7fb4e26a4300 10 monclient(hunting): picked mon.noname-b con 0x56420f0199a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:37.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.423+0000 7fb4e26a4300 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x56420ee720c0 con 0x56420f102dc0 2026-02-25T15:01:37.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.423+0000 7fb4e26a4300 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x56420ee81a80 con 0x56420f0199a0 2026-02-25T15:01:37.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.423+0000 7fb4e26a4300 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x56420f01ee90 con 0x56420f0195d0 2026-02-25T15:01:37.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.423+0000 7fb4e26a4300 10 monclient(hunting): _renew_subs 2026-02-25T15:01:37.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.423+0000 7fb4e26a4300 10 monclient(hunting): authenticate will time out at 2987.023066s 2026-02-25T15:01:37.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.423+0000 7fb4e095a640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x56420f102dc0 0x56420f10c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44592/0 (socket says 10.20.193.90:44592) 2026-02-25T15:01:37.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.423+0000 7fb4e095a640 1 -- 10.20.193.90:0/2575267358 learned_addr learned my addr 10.20.193.90:0/2575267358 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:37.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.424+0000 7fb4d3fff640 1 -- 10.20.193.90:0/2575267358 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2221837101 0 0) 0x56420ee720c0 con 0x56420f102dc0 2026-02-25T15:01:37.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.424+0000 7fb4d3fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:37.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.424+0000 7fb4d3fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:37.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.424+0000 7fb4d3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:37.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.424+0000 7fb4d3fff640 10 monclient(hunting): my global_id is 4353 2026-02-25T15:01:37.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.424+0000 7fb4d3fff640 1 -- 10.20.193.90:0/2575267358 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4c8001e10 con 0x56420f102dc0 2026-02-25T15:01:37.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.425+0000 7fb4d3fff640 1 -- 10.20.193.90:0/2575267358 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2823883836 0 0) 0x56420ee81a80 con 0x56420f0199a0 2026-02-25T15:01:37.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.425+0000 7fb4d3fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:37.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.425+0000 7fb4d3fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:37.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.425+0000 7fb4d3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:37.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.425+0000 7fb4d3fff640 10 monclient(hunting): my global_id is 4366 2026-02-25T15:01:37.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.425+0000 7fb4d3fff640 1 -- 10.20.193.90:0/2575267358 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x56420ee720c0 con 0x56420f0199a0 2026-02-25T15:01:37.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.425+0000 7fb4d3fff640 1 -- 10.20.193.90:0/2575267358 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3373345588 0 0) 0x56420f01ee90 con 0x56420f0195d0 2026-02-25T15:01:37.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.425+0000 7fb4d3fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:37.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.425+0000 7fb4d3fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:37.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.425+0000 7fb4d3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:37.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.425+0000 7fb4d3fff640 10 monclient(hunting): my global_id is 4358 2026-02-25T15:01:37.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.425+0000 7fb4d3fff640 1 -- 10.20.193.90:0/2575267358 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x56420ee81a80 con 0x56420f0195d0 2026-02-25T15:01:37.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4d3fff640 1 -- 10.20.193.90:0/2575267358 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 794210722 0 0) 0x7fb4c8001e10 con 0x56420f102dc0 2026-02-25T15:01:37.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4d3fff640 1 -- 10.20.193.90:0/2575267358 >> v1:10.20.193.94:6789/0 conn(0x56420f0195d0 legacy=0x56420f1026b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:37.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4d3fff640 1 -- 10.20.193.90:0/2575267358 >> v1:10.20.193.93:6789/0 conn(0x56420f0199a0 legacy=0x56420f171ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:37.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4d3fff640 10 monclient: _finish_hunting 0 2026-02-25T15:01:37.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4d3fff640 1 monclient: found mon.noname-a 2026-02-25T15:01:37.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:37.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4d3fff640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:37.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4d3fff640 1 -- 10.20.193.90:0/2575267358 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56420f01f170 con 0x56420f102dc0 2026-02-25T15:01:37.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4d3fff640 10 monclient: _finish_auth 0 2026-02-25T15:01:37.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4d3fff640 10 monclient: _check_auth_tickets 2026-02-25T15:01:37.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:37.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4d3fff640 1 -- 10.20.193.90:0/2575267358 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb4cc0015f0 con 0x56420f102dc0 2026-02-25T15:01:37.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4d3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:37.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4d3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:37.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4d3fff640 10 monclient: dump: 2026-02-25T15:01:37.437 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:37.437 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:37.437 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:37.437 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:37.438 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:37.438 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:37.438 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:37.438 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:37.438 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:37.438 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:37.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4e26a4300 5 monclient: authenticate success, global_id 4353 2026-02-25T15:01:37.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.426+0000 7fb4e26a4300 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:37.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.427+0000 7fb4d3fff640 1 -- 10.20.193.90:0/2575267358 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb4cc0017c0 con 0x56420f102dc0 2026-02-25T15:01:37.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.427+0000 7fb4d3fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:37.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.427+0000 7fb4e26a4300 10 monclient: get_monmap_and_config success 2026-02-25T15:01:37.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.427+0000 7fb4e26a4300 4 set_mon_vals no callback set 2026-02-25T15:01:37.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.427+0000 7fb4e26a4300 10 monclient: shutdown 2026-02-25T15:01:37.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.427+0000 7fb4e26a4300 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:37.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.427+0000 7fb4d3fff640 1 -- 10.20.193.90:0/2575267358 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb4cc001c00 con 0x56420f102dc0 2026-02-25T15:01:37.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.427+0000 7fb4e26a4300 1 -- 10.20.193.90:0/2575267358 >> v1:10.20.193.90:6789/0 conn(0x56420f102dc0 legacy=0x56420f10c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:37.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.427+0000 7fb4d3fff640 10 monclient: discarding stray monitor message mon_map magic: 0 2026-02-25T15:01:37.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.427+0000 7fb4e26a4300 1 -- 10.20.193.90:0/2575267358 shutdown_connections 2026-02-25T15:01:37.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.427+0000 7fb4e26a4300 1 -- 10.20.193.90:0/2575267358 >> 10.20.193.90:0/2575267358 conn(0x56420f063ed0 msgr2=0x56420f0642d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:37.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.427+0000 7fb4e26a4300 1 -- 10.20.193.90:0/2575267358 shutdown_connections 2026-02-25T15:01:37.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.427+0000 7fb4e26a4300 1 -- 10.20.193.90:0/2575267358 wait complete. 2026-02-25T15:01:37.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.427+0000 7fb4e26a4300 10 monclient: build_initial_monmap 2026-02-25T15:01:37.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.427+0000 7fb4e26a4300 10 monclient: monmap: 2026-02-25T15:01:37.440 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:37.440 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:37.440 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:37.440 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:37.440 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:37.440 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:37.440 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:37.440 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:37.440 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:37.441 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:37.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 1 librados: starting msgr at 2026-02-25T15:01:37.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 1 librados: starting objecter 2026-02-25T15:01:37.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:37.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 1 Processor -- start 2026-02-25T15:01:37.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 1 -- start start 2026-02-25T15:01:37.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 1 librados: setting wanted keys 2026-02-25T15:01:37.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 1 librados: calling monclient init 2026-02-25T15:01:37.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 10 monclient: init 2026-02-25T15:01:37.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 10 monclient: _reopen_session rank -1 2026-02-25T15:01:37.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:37.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 10 monclient(hunting): picked mon.noname-b con 0x56420f0195d0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:37.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 10 monclient(hunting): picked mon.noname-a con 0x56420f0199a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:37.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 10 monclient(hunting): picked mon.noname-c con 0x56420f102dc0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:37.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x56420f05cd20 con 0x56420f0199a0 2026-02-25T15:01:37.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x56420f0c9d70 con 0x56420f0195d0 2026-02-25T15:01:37.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x56420f0ca050 con 0x56420f102dc0 2026-02-25T15:01:37.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 10 monclient(hunting): _renew_subs 2026-02-25T15:01:37.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.428+0000 7fb4e26a4300 10 monclient(hunting): authenticate will time out at 2987.028768s 2026-02-25T15:01:37.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.429+0000 7fb4e115b640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x56420f0195d0 0x56420f05c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46210/0 (socket says 10.20.193.90:46210) 2026-02-25T15:01:37.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.429+0000 7fb4e115b640 1 -- 10.20.193.90:0/1626763299 learned_addr learned my addr 10.20.193.90:0/1626763299 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:37.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.429+0000 7fb4d17fa640 10 client.?.objecter ms_handle_connect 0x56420f0195d0 2026-02-25T15:01:37.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.429+0000 7fb4d17fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:37.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.429+0000 7fb4d17fa640 10 client.?.objecter ms_handle_connect 0x56420f102dc0 2026-02-25T15:01:37.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.429+0000 7fb4d17fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:37.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.429+0000 7fb4d17fa640 10 client.?.objecter ms_handle_connect 0x56420f0199a0 2026-02-25T15:01:37.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.429+0000 7fb4d17fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:37.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 989576281 0 0) 0x56420f0c9d70 con 0x56420f0195d0 2026-02-25T15:01:37.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:37.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:37.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:37.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 10 monclient(hunting): my global_id is 4369 2026-02-25T15:01:37.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4c0001e10 con 0x56420f0195d0 2026-02-25T15:01:37.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2603214852 0 0) 0x56420f0ca050 con 0x56420f102dc0 2026-02-25T15:01:37.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:37.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:37.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:37.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 10 monclient(hunting): my global_id is 4361 2026-02-25T15:01:37.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x56420f0c9d70 con 0x56420f102dc0 2026-02-25T15:01:37.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1511182765 0 0) 0x56420f05cd20 con 0x56420f0199a0 2026-02-25T15:01:37.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:37.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:37.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:37.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 10 monclient(hunting): my global_id is 4356 2026-02-25T15:01:37.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.430+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x56420f0ca050 con 0x56420f0199a0 2026-02-25T15:01:37.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.432+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1639685289 0 0) 0x7fb4c0001e10 con 0x56420f0195d0 2026-02-25T15:01:37.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.432+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x56420f05cd20 con 0x56420f0195d0 2026-02-25T15:01:37.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.432+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb4d80019a0 con 0x56420f0195d0 2026-02-25T15:01:37.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.432+0000 7fb4d17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:37.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.432+0000 7fb4d17fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:37.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.432+0000 7fb4d17fa640 10 monclient(hunting): dump: 2026-02-25T15:01:37.445 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:37.445 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:37.445 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:37.445 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:37.445 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:37.445 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:37.445 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:37.445 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:37.445 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:37.445 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:37.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.432+0000 7fb4d17fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:37.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.433+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1426587082 0 0) 0x56420f0ca050 con 0x56420f0199a0 2026-02-25T15:01:37.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.433+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb4c0001e10 con 0x56420f0199a0 2026-02-25T15:01:37.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.433+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb4cc001950 con 0x56420f0199a0 2026-02-25T15:01:37.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.433+0000 7fb4d17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:37.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.433+0000 7fb4d17fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:37.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.433+0000 7fb4d17fa640 10 monclient(hunting): dump: 2026-02-25T15:01:37.446 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:37.446 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:37.446 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:37.446 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:37.446 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:37.446 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:37.447 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:37.447 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:37.447 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:37.447 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:37.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.433+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2558776762 0 0) 0x56420f0c9d70 con 0x56420f102dc0 2026-02-25T15:01:37.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.433+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x56420f0ca050 con 0x56420f102dc0 2026-02-25T15:01:37.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb4d4001bf0 con 0x56420f102dc0 2026-02-25T15:01:37.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4d17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:37.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4d17fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:37.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4d17fa640 10 monclient(hunting): dump: 2026-02-25T15:01:37.447 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:37.447 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:37.448 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:37.448 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:37.448 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:37.448 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:37.448 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:37.448 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:37.448 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:37.448 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:37.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1935712981 0 0) 0x56420f05cd20 con 0x56420f0195d0 2026-02-25T15:01:37.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 >> v1:10.20.193.94:6789/0 conn(0x56420f102dc0 legacy=0x56420f0e5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:37.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 >> v1:10.20.193.90:6789/0 conn(0x56420f0199a0 legacy=0x56420f0c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:37.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4d17fa640 10 monclient: _finish_hunting 0 2026-02-25T15:01:37.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4d17fa640 1 monclient: found mon.b 2026-02-25T15:01:37.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4d17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:37.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4d17fa640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:37.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56420f0ca330 con 0x56420f0195d0 2026-02-25T15:01:37.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4d17fa640 10 monclient: _finish_auth 0 2026-02-25T15:01:37.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4d17fa640 10 monclient: _check_auth_tickets 2026-02-25T15:01:37.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4d17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:37.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4e26a4300 5 monclient: authenticate success, global_id 4369 2026-02-25T15:01:37.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4e26a4300 10 monclient: _renew_subs 2026-02-25T15:01:37.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4e26a4300 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:37.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4e26a4300 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x56420f0ca610 con 0x56420f0195d0 2026-02-25T15:01:37.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4e26a4300 10 client.4369.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:37.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4e26a4300 10 monclient: _renew_subs 2026-02-25T15:01:37.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4e26a4300 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:37.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4e26a4300 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x56420f0c78f0 con 0x56420f0195d0 2026-02-25T15:01:37.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4e26a4300 1 librados: init done 2026-02-25T15:01:37.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.434+0000 7fb4e26a4300 10 librados: wait_for_osdmap waiting 2026-02-25T15:01:37.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.444+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb4d8001d10 con 0x56420f0195d0 2026-02-25T15:01:37.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.444+0000 7fb4d17fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:37.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.444+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb4d8002350 con 0x56420f0195d0 2026-02-25T15:01:37.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.444+0000 7fb4d17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:37.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.444+0000 7fb4d17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:37.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.444+0000 7fb4d17fa640 10 monclient: dump: 2026-02-25T15:01:37.453 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:37.453 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:37.453 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:37.453 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:37.453 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:37.453 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:37.453 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:37.453 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:37.453 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:37.453 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:37.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.444+0000 7fb4d37fe640 4 set_mon_vals no callback set 2026-02-25T15:01:37.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.445+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fb4d8002530 con 0x56420f0195d0 2026-02-25T15:01:37.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.445+0000 7fb4e095a640 10 monclient: get_auth_request con 0x7fb4c00788a0 auth_method 0 2026-02-25T15:01:37.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.445+0000 7fb4d17fa640 10 client.4369.objecter ms_handle_connect 0x7fb4c00788a0 2026-02-25T15:01:37.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.446+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(25..25 src has 1..25) ==== 8102+0+0 (unknown 2042071001 0 0) 0x7fb4d8094310 con 0x56420f0195d0 2026-02-25T15:01:37.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.446+0000 7fb4d17fa640 10 client.4369.objecter ms_dispatch2 0x56420ef2ed50 osd_map(25..25 src has 1..25) 2026-02-25T15:01:37.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.446+0000 7fb4d17fa640 3 client.4369.objecter handle_osd_map got epochs [25,25] > 0 2026-02-25T15:01:37.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.446+0000 7fb4d17fa640 3 client.4369.objecter handle_osd_map decoding full epoch 25 2026-02-25T15:01:37.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.446+0000 7fb4d17fa640 20 client.4369.objecter dump_active .. 0 homeless 2026-02-25T15:01:37.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.446+0000 7fb4e26a4300 10 librados: wait_for_osdmap done waiting 2026-02-25T15:01:37.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.447+0000 7fb4e26a4300 10 start_mon_command cmd=[{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-25T15:01:37.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.447+0000 7fb4e26a4300 10 monclient: _send_command 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-25T15:01:37.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.447+0000 7fb4e26a4300 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:37.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:37.447+0000 7fb4e26a4300 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) -- 0x56420f0c6430 con 0x56420f0195d0 2026-02-25T15:01:38.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.446+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v26) ==== 141+0+0 (unknown 433528931 0 0) 0x7fb4d805dc70 con 0x56420f0195d0 2026-02-25T15:01:38.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.446+0000 7fb4d17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-25T15:01:38.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.446+0000 7fb4d17fa640 10 monclient: _finish_command 1 = system:0 enabled application 'rbd' on pool 'rbd' 2026-02-25T15:01:38.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.446+0000 7fb4e26a4300 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:38.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.446+0000 7fb4e26a4300 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x56420ef0afb0 con 0x56420f0195d0 2026-02-25T15:01:38.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.448+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (unknown 2370098492 0 0) 0x7fb4d805d340 con 0x56420f0195d0 2026-02-25T15:01:38.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.448+0000 7fb4d17fa640 10 monclient: handle_get_version_reply finishing 1 version 26 2026-02-25T15:01:38.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.448+0000 7fb4d37fe640 10 client.4369.objecter _get_latest_version latest 26, waiting 2026-02-25T15:01:38.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.448+0000 7fb4d37fe640 10 client.4369.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:38.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.448+0000 7fb4d37fe640 10 monclient: _renew_subs 2026-02-25T15:01:38.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.448+0000 7fb4d37fe640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:38.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.448+0000 7fb4d37fe640 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=26}) -- 0x7fb4b8002ae0 con 0x56420f0195d0 2026-02-25T15:01:38.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.1 v1:10.20.193.93:6789/0 11 ==== osd_map(26..26 src has 1..26) ==== 669+0+0 (unknown 3974796500 0 0) 0x7fb4d805ce30 con 0x56420f0195d0 2026-02-25T15:01:38.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4d17fa640 10 client.4369.objecter ms_dispatch2 0x56420ef2ed50 osd_map(26..26 src has 1..26) 2026-02-25T15:01:38.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4d17fa640 3 client.4369.objecter handle_osd_map got epochs [26,26] > 25 2026-02-25T15:01:38.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4d17fa640 3 client.4369.objecter handle_osd_map decoding incremental epoch 26 2026-02-25T15:01:38.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4d17fa640 20 client.4369.objecter dump_active .. 0 homeless 2026-02-25T15:01:38.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4e26a4300 10 librados: create oid=rbd_trash nspace= 2026-02-25T15:01:38.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4e26a4300 10 client.4369.objecter _op_submit op 0x56420f0e68a0 2026-02-25T15:01:38.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4e26a4300 20 client.4369.objecter _calc_target epoch 26 base rbd_trash @2 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:01:38.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4e26a4300 20 client.4369.objecter _calc_target target rbd_trash @2 -> pgid 2.5a749323 2026-02-25T15:01:38.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4e26a4300 10 client.4369.objecter _calc_target raw pgid 2.5a749323 -> actual 2.3 acting [11,5] primary 11 2026-02-25T15:01:38.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4e26a4300 20 client.4369.objecter _get_session s=0x56420ee8b280 osd=11 3 2026-02-25T15:01:38.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4e26a4300 10 client.4369.objecter _op_submit oid rbd_trash '@2' '@2' [create] tid 1 osd.11 2026-02-25T15:01:38.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4e26a4300 20 client.4369.objecter get_session s=0x56420ee8b280 osd=11 3 2026-02-25T15:01:38.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4e26a4300 15 client.4369.objecter _session_op_assign 11 1 2026-02-25T15:01:38.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4e26a4300 15 client.4369.objecter _send_op 1 to 2.3 on osd.11 2026-02-25T15:01:38.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4e26a4300 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.189:6801/4031575111 -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26) -- 0x56420f0e8d50 con 0x56420f0e7260 2026-02-25T15:01:38.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4e26a4300 20 client.4369.objecter put_session s=0x56420ee8b280 osd=11 4 2026-02-25T15:01:38.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.449+0000 7fb4e26a4300 5 client.4369.objecter 1 in flight 2026-02-25T15:01:38.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.450+0000 7fb4e23e4640 10 monclient: get_auth_request con 0x56420f0e7260 auth_method 0 2026-02-25T15:01:38.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:38.451+0000 7fb4d17fa640 10 client.4369.objecter ms_handle_connect 0x56420f0e7260 2026-02-25T15:01:39.049 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.048+0000 7fb4e23e4640 1 -- 10.20.193.90:0/1626763299 <== osd.11 v1:10.20.193.189:6801/4031575111 1 ==== osd_op_reply(1 rbd_trash [create] v26'1 uv1 ondisk = 0) ==== 153+0+0 (unknown 504360132 0 0) 0x7fb4d4003320 con 0x56420f0e7260 2026-02-25T15:01:39.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.048+0000 7fb4e23e4640 10 client.4369.objecter ms_dispatch2 0x56420ef2ed50 osd_op_reply(1 rbd_trash [create] v26'1 uv1 ondisk = 0) 2026-02-25T15:01:39.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.048+0000 7fb4e23e4640 10 client.4369.objecter in handle_osd_op_reply 2026-02-25T15:01:39.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.048+0000 7fb4e23e4640 7 client.4369.objecter handle_osd_op_reply 1 ondisk uv 1 in 2.3 attempt 0 2026-02-25T15:01:39.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.048+0000 7fb4e23e4640 10 client.4369.objecter op 0 rval 0 len 0 2026-02-25T15:01:39.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.048+0000 7fb4e23e4640 15 client.4369.objecter handle_osd_op_reply completed tid 1 2026-02-25T15:01:39.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.048+0000 7fb4e23e4640 15 client.4369.objecter _finish_op 1 2026-02-25T15:01:39.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.048+0000 7fb4e23e4640 20 client.4369.objecter put_session s=0x56420ee8b280 osd=11 4 2026-02-25T15:01:39.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.048+0000 7fb4e23e4640 15 client.4369.objecter _session_op_remove 11 1 2026-02-25T15:01:39.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.048+0000 7fb4e23e4640 5 client.4369.objecter 0 in flight 2026-02-25T15:01:39.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.048+0000 7fb4e26a4300 10 librados: Objecter returned from create r=0 2026-02-25T15:01:39.051 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.049+0000 7fb4e26a4300 10 client.4369.objecter _op_submit op 0x56420f1de760 2026-02-25T15:01:39.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.049+0000 7fb4e26a4300 20 client.4369.objecter _calc_target epoch 26 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:01:39.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.049+0000 7fb4e26a4300 20 client.4369.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-25T15:01:39.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.049+0000 7fb4e26a4300 10 client.4369.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [11,13] primary 11 2026-02-25T15:01:39.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.049+0000 7fb4e26a4300 10 client.4369.objecter chose random osd.11 of [11,13] 2026-02-25T15:01:39.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.049+0000 7fb4e26a4300 20 client.4369.objecter _get_session s=0x56420ee8b280 osd=11 3 2026-02-25T15:01:39.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.049+0000 7fb4e26a4300 10 client.4369.objecter _op_submit oid rbd_info '@2' '@2' [call rbd.metadata_list in=33b] tid 2 osd.11 2026-02-25T15:01:39.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.049+0000 7fb4e26a4300 20 client.4369.objecter get_session s=0x56420ee8b280 osd=11 3 2026-02-25T15:01:39.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.049+0000 7fb4e26a4300 15 client.4369.objecter _session_op_assign 11 2 2026-02-25T15:01:39.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.049+0000 7fb4e26a4300 15 client.4369.objecter _send_op 2 to 2.2 on osd.11 2026-02-25T15:01:39.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.049+0000 7fb4e26a4300 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.189:6801/4031575111 -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e26) -- 0x56420f1dec90 con 0x56420f0e7260 2026-02-25T15:01:39.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.049+0000 7fb4e26a4300 20 client.4369.objecter put_session s=0x56420ee8b280 osd=11 4 2026-02-25T15:01:39.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.049+0000 7fb4e26a4300 5 client.4369.objecter 1 in flight 2026-02-25T15:01:39.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e23e4640 1 -- 10.20.193.90:0/1626763299 <== osd.11 v1:10.20.193.189:6801/4031575111 2 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (unknown 3560438577 0 0) 0x7fb4d4003320 con 0x56420f0e7260 2026-02-25T15:01:39.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e23e4640 10 client.4369.objecter ms_dispatch2 0x56420ef2ed50 osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:01:39.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e23e4640 10 client.4369.objecter in handle_osd_op_reply 2026-02-25T15:01:39.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e23e4640 7 client.4369.objecter handle_osd_op_reply 2 ondisk uv 0 in 2.2 attempt 0 2026-02-25T15:01:39.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e23e4640 10 client.4369.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:01:39.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e23e4640 10 client.4369.objecter op 0 rval 0 len 0 2026-02-25T15:01:39.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e23e4640 15 client.4369.objecter handle_osd_op_reply completed tid 2 2026-02-25T15:01:39.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e23e4640 15 client.4369.objecter _finish_op 2 2026-02-25T15:01:39.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e23e4640 20 client.4369.objecter put_session s=0x56420ee8b280 osd=11 4 2026-02-25T15:01:39.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e23e4640 15 client.4369.objecter _session_op_remove 11 2 2026-02-25T15:01:39.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e23e4640 5 client.4369.objecter 0 in flight 2026-02-25T15:01:39.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e26a4300 10 client.4369.objecter _op_submit op 0x56420f1dd8c0 2026-02-25T15:01:39.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e26a4300 20 client.4369.objecter _calc_target epoch 26 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:01:39.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e26a4300 20 client.4369.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-25T15:01:39.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e26a4300 10 client.4369.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [11,13] primary 11 2026-02-25T15:01:39.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e26a4300 10 client.4369.objecter chose random osd.11 of [11,13] 2026-02-25T15:01:39.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e26a4300 20 client.4369.objecter _get_session s=0x56420ee8b280 osd=11 3 2026-02-25T15:01:39.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e26a4300 10 client.4369.objecter _op_submit oid rbd_info '@2' '@2' [read 0~0] tid 3 osd.11 2026-02-25T15:01:39.068 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e26a4300 20 client.4369.objecter get_session s=0x56420ee8b280 osd=11 3 2026-02-25T15:01:39.068 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e26a4300 15 client.4369.objecter _session_op_assign 11 3 2026-02-25T15:01:39.068 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e26a4300 15 client.4369.objecter _send_op 3 to 2.2 on osd.11 2026-02-25T15:01:39.068 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e26a4300 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.189:6801/4031575111 -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e26) -- 0x56420f1ddd70 con 0x56420f0e7260 2026-02-25T15:01:39.068 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e26a4300 20 client.4369.objecter put_session s=0x56420ee8b280 osd=11 4 2026-02-25T15:01:39.068 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.064+0000 7fb4e26a4300 5 client.4369.objecter 1 in flight 2026-02-25T15:01:39.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.068+0000 7fb4e23e4640 1 -- 10.20.193.90:0/1626763299 <== osd.11 v1:10.20.193.189:6801/4031575111 3 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (unknown 3580297533 0 0) 0x7fb4d4003320 con 0x56420f0e7260 2026-02-25T15:01:39.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.068+0000 7fb4e23e4640 10 client.4369.objecter ms_dispatch2 0x56420ef2ed50 osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:01:39.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.068+0000 7fb4e23e4640 10 client.4369.objecter in handle_osd_op_reply 2026-02-25T15:01:39.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.068+0000 7fb4e23e4640 7 client.4369.objecter handle_osd_op_reply 3 ondisk uv 0 in 2.2 attempt 0 2026-02-25T15:01:39.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.068+0000 7fb4e23e4640 10 client.4369.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:01:39.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.068+0000 7fb4e23e4640 10 client.4369.objecter op 0 rval 0 len 0 2026-02-25T15:01:39.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.068+0000 7fb4e23e4640 15 client.4369.objecter handle_osd_op_reply completed tid 3 2026-02-25T15:01:39.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.068+0000 7fb4e23e4640 15 client.4369.objecter _finish_op 3 2026-02-25T15:01:39.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.068+0000 7fb4e23e4640 20 client.4369.objecter put_session s=0x56420ee8b280 osd=11 4 2026-02-25T15:01:39.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.068+0000 7fb4e23e4640 15 client.4369.objecter _session_op_remove 11 3 2026-02-25T15:01:39.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.068+0000 7fb4e23e4640 5 client.4369.objecter 0 in flight 2026-02-25T15:01:39.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.068+0000 7fb4d2ffd640 10 client.4369.objecter allocate_selfmanaged_snap; pool: 2 2026-02-25T15:01:39.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.068+0000 7fb4d2ffd640 10 client.4369.objecter pool_op_submit 4 2026-02-25T15:01:39.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.068+0000 7fb4d2ffd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:39.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.068+0000 7fb4d2ffd640 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.93:6789/0 -- pool_op(create unmanaged snap pool 2 tid 4 name v0) -- 0x7fb4bc003110 con 0x56420f0195d0 2026-02-25T15:01:39.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.525+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.1 v1:10.20.193.93:6789/0 12 ==== pool_op_reply(tid 4 (0) Success v27) ==== 55+0+0 (unknown 803611251 0 0) 0x7fb4d8002350 con 0x56420f0195d0 2026-02-25T15:01:39.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.525+0000 7fb4d17fa640 10 client.4369.objecter ms_dispatch2 0x56420ef2ed50 pool_op_reply(tid 4 (0) Success v27) 2026-02-25T15:01:39.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.525+0000 7fb4d17fa640 10 client.4369.objecter handle_pool_op_reply pool_op_reply(tid 4 (0) Success v27) 2026-02-25T15:01:39.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.525+0000 7fb4d17fa640 10 client.4369.objecter have request 4 at 0x7fb4bc0030a0 Op: create unmanaged snap 2026-02-25T15:01:39.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.525+0000 7fb4d17fa640 20 client.4369.objecter waiting for client to reach epoch 27 before calling back 2026-02-25T15:01:39.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.525+0000 7fb4d17fa640 10 client.4369.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:39.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.525+0000 7fb4d17fa640 10 monclient: _renew_subs 2026-02-25T15:01:39.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.525+0000 7fb4d17fa640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:39.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.525+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=27}) -- 0x56420f05cd20 con 0x56420f0195d0 2026-02-25T15:01:39.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.525+0000 7fb4d17fa640 10 client.4369.objecter done 2026-02-25T15:01:39.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.1 v1:10.20.193.93:6789/0 13 ==== osd_map(27..27 src has 1..27) ==== 629+0+0 (unknown 1020006130 0 0) 0x7fb4d805d050 con 0x56420f0195d0 2026-02-25T15:01:39.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d17fa640 10 client.4369.objecter ms_dispatch2 0x56420ef2ed50 osd_map(27..27 src has 1..27) 2026-02-25T15:01:39.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d17fa640 3 client.4369.objecter handle_osd_map got epochs [27,27] > 26 2026-02-25T15:01:39.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d17fa640 3 client.4369.objecter handle_osd_map decoding incremental epoch 27 2026-02-25T15:01:39.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d17fa640 20 client.4369.objecter dump_active .. 0 homeless 2026-02-25T15:01:39.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d37fe640 20 librados: queue_aio_write 0x56420f1d9cf0 completion 0x56420f10d1f0 write_seq 1 2026-02-25T15:01:39.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d37fe640 10 client.4369.objecter _op_submit op 0x7fb4b8006420 2026-02-25T15:01:39.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d37fe640 20 client.4369.objecter _calc_target epoch 27 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:01:39.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d37fe640 20 client.4369.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-25T15:01:39.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d37fe640 10 client.4369.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [11,13] primary 11 2026-02-25T15:01:39.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d37fe640 20 client.4369.objecter _get_session s=0x56420ee8b280 osd=11 3 2026-02-25T15:01:39.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d37fe640 10 client.4369.objecter _op_submit oid rbd_info '@2' '@2' [create,write 0~8 in=8b] tid 5 osd.11 2026-02-25T15:01:39.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d37fe640 20 client.4369.objecter get_session s=0x56420ee8b280 osd=11 3 2026-02-25T15:01:39.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d37fe640 15 client.4369.objecter _session_op_assign 11 5 2026-02-25T15:01:39.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d37fe640 15 client.4369.objecter _send_op 5 to 2.2 on osd.11 2026-02-25T15:01:39.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d37fe640 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.189:6801/4031575111 -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x7fb4b8006930 con 0x56420f0e7260 2026-02-25T15:01:39.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d37fe640 20 client.4369.objecter put_session s=0x56420ee8b280 osd=11 4 2026-02-25T15:01:39.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.526+0000 7fb4d37fe640 5 client.4369.objecter 1 in flight 2026-02-25T15:01:39.650 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4e23e4640 1 -- 10.20.193.90:0/1626763299 <== osd.11 v1:10.20.193.189:6801/4031575111 4 ==== osd_op_reply(5 rbd_info [create,write 0~8] v27'1 uv1 ondisk = 0) ==== 194+0+0 (unknown 3272180499 0 0) 0x7fb4d4003320 con 0x56420f0e7260 2026-02-25T15:01:39.651 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4e23e4640 10 client.4369.objecter ms_dispatch2 0x56420ef2ed50 osd_op_reply(5 rbd_info [create,write 0~8] v27'1 uv1 ondisk = 0) 2026-02-25T15:01:39.651 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4e23e4640 10 client.4369.objecter in handle_osd_op_reply 2026-02-25T15:01:39.651 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4e23e4640 7 client.4369.objecter handle_osd_op_reply 5 ondisk uv 1 in 2.2 attempt 0 2026-02-25T15:01:39.651 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4e23e4640 10 client.4369.objecter op 0 rval 0 len 0 2026-02-25T15:01:39.651 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4e23e4640 10 client.4369.objecter op 1 rval 0 len 0 2026-02-25T15:01:39.651 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4e23e4640 15 client.4369.objecter handle_osd_op_reply completed tid 5 2026-02-25T15:01:39.651 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4e23e4640 15 client.4369.objecter _finish_op 5 2026-02-25T15:01:39.651 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4e23e4640 20 client.4369.objecter put_session s=0x56420ee8b280 osd=11 4 2026-02-25T15:01:39.651 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4e23e4640 15 client.4369.objecter _session_op_remove 11 5 2026-02-25T15:01:39.652 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4e23e4640 5 client.4369.objecter 0 in flight 2026-02-25T15:01:39.652 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4e23e4640 20 librados: complete_aio_write 0x56420f10d1f0 2026-02-25T15:01:39.652 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4d2ffd640 10 client.4369.objecter delete_selfmanaged_snap; pool: 2; snap: 2 2026-02-25T15:01:39.652 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4d2ffd640 10 client.4369.objecter pool_op_submit 6 2026-02-25T15:01:39.652 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4d2ffd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:39.652 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:39.648+0000 7fb4d2ffd640 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.93:6789/0 -- pool_op(delete unmanaged snap pool 2 tid 6 name v27) -- 0x7fb4bc0033b0 con 0x56420f0195d0 2026-02-25T15:01:40.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.429+0000 7fb4c7fff640 10 monclient: tick 2026-02-25T15:01:40.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.429+0000 7fb4c7fff640 10 monclient: _check_auth_tickets 2026-02-25T15:01:40.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.429+0000 7fb4c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:40.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.429+0000 7fb4c7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:40.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.531+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.1 v1:10.20.193.93:6789/0 14 ==== pool_op_reply(tid 6 (0) Success v28) ==== 43+0+0 (unknown 4038710169 0 0) 0x7fb4d805d570 con 0x56420f0195d0 2026-02-25T15:01:40.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.531+0000 7fb4d17fa640 10 client.4369.objecter ms_dispatch2 0x56420ef2ed50 pool_op_reply(tid 6 (0) Success v28) 2026-02-25T15:01:40.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.531+0000 7fb4d17fa640 10 client.4369.objecter handle_pool_op_reply pool_op_reply(tid 6 (0) Success v28) 2026-02-25T15:01:40.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.532+0000 7fb4d17fa640 10 client.4369.objecter have request 6 at 0x7fb4bc0030a0 Op: delete unmanaged snap 2026-02-25T15:01:40.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.532+0000 7fb4d17fa640 20 client.4369.objecter waiting for client to reach epoch 28 before calling back 2026-02-25T15:01:40.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.532+0000 7fb4d17fa640 10 client.4369.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:40.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.532+0000 7fb4d17fa640 10 monclient: _renew_subs 2026-02-25T15:01:40.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.532+0000 7fb4d17fa640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:40.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.532+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=28}) -- 0x56420f0c9d70 con 0x56420f0195d0 2026-02-25T15:01:40.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.532+0000 7fb4d17fa640 10 client.4369.objecter done 2026-02-25T15:01:40.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d17fa640 1 -- 10.20.193.90:0/1626763299 <== mon.1 v1:10.20.193.93:6789/0 15 ==== osd_map(28..28 src has 1..28) ==== 657+0+0 (unknown 1903122315 0 0) 0x7fb4d805b270 con 0x56420f0195d0 2026-02-25T15:01:40.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d17fa640 10 client.4369.objecter ms_dispatch2 0x56420ef2ed50 osd_map(28..28 src has 1..28) 2026-02-25T15:01:40.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d17fa640 3 client.4369.objecter handle_osd_map got epochs [28,28] > 27 2026-02-25T15:01:40.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d17fa640 3 client.4369.objecter handle_osd_map decoding incremental epoch 28 2026-02-25T15:01:40.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d17fa640 20 client.4369.objecter dump_active .. 0 homeless 2026-02-25T15:01:40.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d37fe640 20 librados: queue_aio_write 0x56420f1d9cf0 completion 0x7fb4bc001da0 write_seq 2 2026-02-25T15:01:40.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d37fe640 10 client.4369.objecter _op_submit op 0x7fb4b8006420 2026-02-25T15:01:40.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d37fe640 20 client.4369.objecter _calc_target epoch 28 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:01:40.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d37fe640 20 client.4369.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-25T15:01:40.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d37fe640 10 client.4369.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [11,13] primary 11 2026-02-25T15:01:40.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d37fe640 20 client.4369.objecter _get_session s=0x56420ee8b280 osd=11 3 2026-02-25T15:01:40.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d37fe640 10 client.4369.objecter _op_submit oid rbd_info '@2' '@2' [write 0~19 in=19b] tid 7 osd.11 2026-02-25T15:01:40.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d37fe640 20 client.4369.objecter get_session s=0x56420ee8b280 osd=11 3 2026-02-25T15:01:40.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d37fe640 15 client.4369.objecter _session_op_assign 11 7 2026-02-25T15:01:40.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d37fe640 15 client.4369.objecter _send_op 7 to 2.2 on osd.11 2026-02-25T15:01:40.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d37fe640 1 -- 10.20.193.90:0/1626763299 --> v1:10.20.193.189:6801/4031575111 -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) -- 0x7fb4b8006e60 con 0x56420f0e7260 2026-02-25T15:01:40.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d37fe640 20 client.4369.objecter put_session s=0x56420ee8b280 osd=11 4 2026-02-25T15:01:40.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.533+0000 7fb4d37fe640 5 client.4369.objecter 1 in flight 2026-02-25T15:01:40.627 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e23e4640 1 -- 10.20.193.90:0/1626763299 <== osd.11 v1:10.20.193.189:6801/4031575111 5 ==== osd_op_reply(7 rbd_info [write 0~19] v28'2 uv2 ondisk = 0) ==== 152+0+0 (unknown 1447781391 0 0) 0x7fb4d4003320 con 0x56420f0e7260 2026-02-25T15:01:40.627 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e23e4640 10 client.4369.objecter ms_dispatch2 0x56420ef2ed50 osd_op_reply(7 rbd_info [write 0~19] v28'2 uv2 ondisk = 0) 2026-02-25T15:01:40.627 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e23e4640 10 client.4369.objecter in handle_osd_op_reply 2026-02-25T15:01:40.627 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e23e4640 7 client.4369.objecter handle_osd_op_reply 7 ondisk uv 2 in 2.2 attempt 0 2026-02-25T15:01:40.627 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e23e4640 10 client.4369.objecter op 0 rval 0 len 0 2026-02-25T15:01:40.627 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e23e4640 15 client.4369.objecter handle_osd_op_reply completed tid 7 2026-02-25T15:01:40.627 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e23e4640 15 client.4369.objecter _finish_op 7 2026-02-25T15:01:40.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e23e4640 20 client.4369.objecter put_session s=0x56420ee8b280 osd=11 4 2026-02-25T15:01:40.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e23e4640 15 client.4369.objecter _session_op_remove 11 7 2026-02-25T15:01:40.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e23e4640 5 client.4369.objecter 0 in flight 2026-02-25T15:01:40.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e23e4640 20 librados: complete_aio_write 0x7fb4bc001da0 2026-02-25T15:01:40.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 10 librados: watch_flush enter 2026-02-25T15:01:40.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 10 librados: watch_flush exit 2026-02-25T15:01:40.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 10 client.4369.objecter close_session for osd.11 2026-02-25T15:01:40.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 1 -- 10.20.193.90:0/1626763299 >> v1:10.20.193.189:6801/4031575111 conn(0x56420f0e7260 legacy=0x56420f0e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:40.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 20 client.4369.objecter put_session s=0x56420ee8b280 osd=11 1 2026-02-25T15:01:40.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 20 client.4369.objecter shutdown clearing up homeless session... 2026-02-25T15:01:40.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 10 client.4369.objecter successfully canceled tick 2026-02-25T15:01:40.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 1 -- 10.20.193.90:0/1626763299 >> v1:10.20.193.189:6800/3647375982 conn(0x7fb4c00788a0 legacy=0x7fb4c0098d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:40.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 10 monclient: shutdown 2026-02-25T15:01:40.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:40.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 1 -- 10.20.193.90:0/1626763299 >> v1:10.20.193.93:6789/0 conn(0x56420f0195d0 legacy=0x56420f05c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:40.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e23e4640 1 -- 10.20.193.90:0/1626763299 reap_dead start 2026-02-25T15:01:40.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 1 -- 10.20.193.90:0/1626763299 shutdown_connections 2026-02-25T15:01:40.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 1 -- 10.20.193.90:0/1626763299 >> 10.20.193.90:0/1626763299 conn(0x56420f063ed0 msgr2=0x56420f102480 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:40.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 1 -- 10.20.193.90:0/1626763299 shutdown_connections 2026-02-25T15:01:40.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 1 -- 10.20.193.90:0/1626763299 wait complete. 2026-02-25T15:01:40.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.625+0000 7fb4e26a4300 1 librados: shutdown 2026-02-25T15:01:40.630 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-25T15:01:40.632 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-25T15:01:40.632 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-25T15:01:40.739 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.737+0000 7f7ac675c640 10 monclient: get_monmap_and_config 2026-02-25T15:01:40.739 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.737+0000 7f7ac675c640 10 monclient: build_initial_monmap 2026-02-25T15:01:40.739 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.737+0000 7f7ac675c640 10 monclient: monmap: 2026-02-25T15:01:40.739 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:40.739 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:40.739 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:40.738157+0000 2026-02-25T15:01:40.740 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:40.738157+0000 2026-02-25T15:01:40.740 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:40.740 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:40.740 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:40.740 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:40.740 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:40.740 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:40.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.737+0000 7f7ac675c640 1 Processor -- start 2026-02-25T15:01:40.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.738+0000 7f7ac675c640 1 -- start start 2026-02-25T15:01:40.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.738+0000 7f7ac675c640 10 monclient: init 2026-02-25T15:01:40.743 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.738+0000 7f7ac675c640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:40.743 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.738+0000 7f7ac675c640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:40.743 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.738+0000 7f7ac675c640 10 monclient(hunting): picked mon.noname-b con 0x7f7ac0157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:40.743 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.738+0000 7f7ac675c640 10 monclient(hunting): picked mon.noname-a con 0x7f7ac0157b50 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:40.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.738+0000 7f7ac675c640 10 monclient(hunting): picked mon.noname-c con 0x7f7ac0164d00 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:40.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.738+0000 7f7ac675c640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac005de70 con 0x7f7ac0157b50 2026-02-25T15:01:40.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.738+0000 7f7ac675c640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac0055b20 con 0x7f7ac0157780 2026-02-25T15:01:40.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.738+0000 7f7ac675c640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac0055e00 con 0x7f7ac0164d00 2026-02-25T15:01:40.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.738+0000 7f7ac675c640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:40.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.738+0000 7f7ac675c640 10 monclient(hunting): authenticate will time out at 2990.338502s 2026-02-25T15:01:40.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.738+0000 7f7abf7fe640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f7ac0157b50 0x7f7ac01645f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44618/0 (socket says 10.20.193.90:44618) 2026-02-25T15:01:40.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.738+0000 7f7abf7fe640 1 -- 10.20.193.90:0/632496831 learned_addr learned my addr 10.20.193.90:0/632496831 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:40.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 1 -- 10.20.193.90:0/632496831 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1213181221 0 0) 0x7f7ac0055b20 con 0x7f7ac0157780 2026-02-25T15:01:40.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:40.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:40.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:40.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 10 monclient(hunting): my global_id is 4372 2026-02-25T15:01:40.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 1 -- 10.20.193.90:0/632496831 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7aa4003610 con 0x7f7ac0157780 2026-02-25T15:01:40.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 1 -- 10.20.193.90:0/632496831 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1027539703 0 0) 0x7f7ac005de70 con 0x7f7ac0157b50 2026-02-25T15:01:40.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:40.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:40.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:40.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 10 monclient(hunting): my global_id is 4359 2026-02-25T15:01:40.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 1 -- 10.20.193.90:0/632496831 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ac0055b20 con 0x7f7ac0157b50 2026-02-25T15:01:40.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 1 -- 10.20.193.90:0/632496831 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1880559647 0 0) 0x7f7ac0055e00 con 0x7f7ac0164d00 2026-02-25T15:01:40.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:40.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:40.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:40.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 10 monclient(hunting): my global_id is 4364 2026-02-25T15:01:40.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.740+0000 7f7abeffd640 1 -- 10.20.193.90:0/632496831 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ac005de70 con 0x7f7ac0164d00 2026-02-25T15:01:40.751 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7abeffd640 1 -- 10.20.193.90:0/632496831 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3839215107 0 0) 0x7f7ac0055b20 con 0x7f7ac0157b50 2026-02-25T15:01:40.751 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7abeffd640 1 -- 10.20.193.90:0/632496831 >> v1:10.20.193.94:6789/0 conn(0x7f7ac0164d00 legacy=0x7f7ac01c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:40.751 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7abeffd640 1 -- 10.20.193.90:0/632496831 >> v1:10.20.193.93:6789/0 conn(0x7f7ac0157780 legacy=0x7f7ac0056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:40.751 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7abeffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:40.751 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7abeffd640 1 monclient: found mon.noname-a 2026-02-25T15:01:40.751 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7abeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:40.751 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7abeffd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:40.751 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7abeffd640 1 -- 10.20.193.90:0/632496831 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac00560e0 con 0x7f7ac0157b50 2026-02-25T15:01:40.752 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7abeffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:40.752 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7abeffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:40.752 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7abeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:40.752 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7abeffd640 1 -- 10.20.193.90:0/632496831 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7ab0002df0 con 0x7f7ac0157b50 2026-02-25T15:01:40.752 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7abeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:40.752 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7abeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:40.752 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7abeffd640 10 monclient: dump: 2026-02-25T15:01:40.752 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:40.753 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:40.753 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:40.753 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:40.753 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:40.753 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:40.753 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:40.753 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:40.753 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:40.753 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:40.753 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7ac675c640 5 monclient: authenticate success, global_id 4359 2026-02-25T15:01:40.753 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.742+0000 7f7ac675c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:40.753 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.744+0000 7f7abeffd640 1 -- 10.20.193.90:0/632496831 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7ab0002fc0 con 0x7f7ac0157b50 2026-02-25T15:01:40.754 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.744+0000 7f7abeffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:40.754 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.744+0000 7f7abeffd640 1 -- 10.20.193.90:0/632496831 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7ab0003400 con 0x7f7ac0157b50 2026-02-25T15:01:40.754 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.744+0000 7f7ac675c640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:40.754 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.744+0000 7f7ac675c640 4 set_mon_vals no callback set 2026-02-25T15:01:40.754 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.744+0000 7f7abeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:40.754 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.744+0000 7f7abeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:40.754 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.744+0000 7f7abeffd640 10 monclient: dump: 2026-02-25T15:01:40.754 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:40.754 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:40.754 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:40.754 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:40.755 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:40.755 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:40.755 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:40.755 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:40.755 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:40.755 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:40.755 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.744+0000 7f7ac675c640 10 monclient: shutdown 2026-02-25T15:01:40.755 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.744+0000 7f7ac675c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:40.755 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.744+0000 7f7ac675c640 1 -- 10.20.193.90:0/632496831 >> v1:10.20.193.90:6789/0 conn(0x7f7ac0157b50 legacy=0x7f7ac01645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:40.755 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.744+0000 7f7ac675c640 1 -- 10.20.193.90:0/632496831 shutdown_connections 2026-02-25T15:01:40.755 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.744+0000 7f7ac675c640 1 -- 10.20.193.90:0/632496831 >> 10.20.193.90:0/632496831 conn(0x7f7ac00b2ef0 msgr2=0x7f7ac00b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:40.755 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.744+0000 7f7ac675c640 1 -- 10.20.193.90:0/632496831 shutdown_connections 2026-02-25T15:01:40.756 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.744+0000 7f7ac675c640 1 -- 10.20.193.90:0/632496831 wait complete. 2026-02-25T15:01:40.756 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 10 monclient: build_initial_monmap 2026-02-25T15:01:40.756 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 10 monclient: monmap: 2026-02-25T15:01:40.756 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:40.756 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:40.756 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:40.756 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:40.756 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:40.756 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:40.756 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:40.756 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:40.756 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:40.757 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:40.757 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 1 librados: starting msgr at 2026-02-25T15:01:40.757 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 1 librados: starting objecter 2026-02-25T15:01:40.757 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:40.757 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 1 Processor -- start 2026-02-25T15:01:40.757 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 1 -- start start 2026-02-25T15:01:40.757 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 1 librados: setting wanted keys 2026-02-25T15:01:40.757 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 1 librados: calling monclient init 2026-02-25T15:01:40.757 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 10 monclient: init 2026-02-25T15:01:40.757 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:40.757 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:40.757 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 10 monclient(hunting): picked mon.noname-a con 0x7f7ac0157780 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:40.758 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 10 monclient(hunting): picked mon.noname-b con 0x7f7ac0157b50 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:40.758 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 10 monclient(hunting): picked mon.noname-c con 0x7f7ac0164d00 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:40.758 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac015d6d0 con 0x7f7ac0157780 2026-02-25T15:01:40.758 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac015d9b0 con 0x7f7ac0157b50 2026-02-25T15:01:40.758 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac015dc90 con 0x7f7ac0164d00 2026-02-25T15:01:40.758 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:40.758 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.745+0000 7f7ac675c640 10 monclient(hunting): authenticate will time out at 2990.345849s 2026-02-25T15:01:40.758 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.746+0000 7f7abffff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f7ac0157780 0x7f7ac015ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44632/0 (socket says 10.20.193.90:44632) 2026-02-25T15:01:40.758 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.746+0000 7f7abffff640 1 -- 10.20.193.90:0/3238999154 learned_addr learned my addr 10.20.193.90:0/3238999154 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:40.758 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.746+0000 7f7abcff9640 10 client.?.objecter ms_handle_connect 0x7f7ac0157780 2026-02-25T15:01:40.758 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.746+0000 7f7abcff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:40.758 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.746+0000 7f7abcff9640 10 client.?.objecter ms_handle_connect 0x7f7ac0157b50 2026-02-25T15:01:40.759 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.746+0000 7f7abcff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:40.759 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.747+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1920386809 0 0) 0x7f7ac015d6d0 con 0x7f7ac0157780 2026-02-25T15:01:40.759 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.747+0000 7f7abcff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:40.759 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.747+0000 7f7abcff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:40.759 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.747+0000 7f7abcff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:40.759 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.747+0000 7f7abcff9640 10 monclient(hunting): my global_id is 4362 2026-02-25T15:01:40.759 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.747+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a90003610 con 0x7f7ac0157780 2026-02-25T15:01:40.759 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.747+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 709790108 0 0) 0x7f7ac015d9b0 con 0x7f7ac0157b50 2026-02-25T15:01:40.759 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.747+0000 7f7abcff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:40.759 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.747+0000 7f7abcff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:40.759 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.747+0000 7f7abcff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:40.759 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.747+0000 7f7abcff9640 10 monclient(hunting): my global_id is 4375 2026-02-25T15:01:40.760 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.747+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ac015d6d0 con 0x7f7ac0157b50 2026-02-25T15:01:40.760 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.747+0000 7f7abcff9640 10 client.?.objecter ms_handle_connect 0x7f7ac0164d00 2026-02-25T15:01:40.760 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.747+0000 7f7abcff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:40.760 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.748+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 51959359 0 0) 0x7f7ac015dc90 con 0x7f7ac0164d00 2026-02-25T15:01:40.760 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.748+0000 7f7abcff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:40.760 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.748+0000 7f7abcff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:40.760 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.748+0000 7f7abcff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:40.760 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.748+0000 7f7abcff9640 10 monclient(hunting): my global_id is 4367 2026-02-25T15:01:40.760 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.748+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ac015d9b0 con 0x7f7ac0164d00 2026-02-25T15:01:40.760 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.749+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1207766096 0 0) 0x7f7a90003610 con 0x7f7ac0157780 2026-02-25T15:01:40.760 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.749+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7ac015dc90 con 0x7f7ac0157780 2026-02-25T15:01:40.760 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.749+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7aac0030f0 con 0x7f7ac0157780 2026-02-25T15:01:40.761 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.749+0000 7f7abcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:40.761 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.749+0000 7f7abcff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:40.761 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.749+0000 7f7abcff9640 10 monclient(hunting): dump: 2026-02-25T15:01:40.761 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:40.761 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:40.761 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:40.761 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:40.761 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:40.761 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:40.761 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:40.761 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:40.761 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:40.762 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:40.762 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.749+0000 7f7abcff9640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:40.764 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.750+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 673364704 0 0) 0x7f7ac015d6d0 con 0x7f7ac0157b50 2026-02-25T15:01:40.764 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.750+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7a90003610 con 0x7f7ac0157b50 2026-02-25T15:01:40.764 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.750+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7ab0003310 con 0x7f7ac0157b50 2026-02-25T15:01:40.764 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.750+0000 7f7abcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:40.764 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.750+0000 7f7abcff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:40.764 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.750+0000 7f7abcff9640 10 monclient(hunting): dump: 2026-02-25T15:01:40.764 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:40.765 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:40.765 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:40.765 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:40.765 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:40.765 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:40.765 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:40.765 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:40.765 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:40.765 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:40.765 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1187394962 0 0) 0x7f7ac015dc90 con 0x7f7ac0157780 2026-02-25T15:01:40.765 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 >> v1:10.20.193.94:6789/0 conn(0x7f7ac0164d00 legacy=0x7f7ac0147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:40.766 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 >> v1:10.20.193.93:6789/0 conn(0x7f7ac0157b50 legacy=0x7f7ac0146740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:40.766 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7abcff9640 10 monclient: _finish_hunting 0 2026-02-25T15:01:40.766 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7abcff9640 1 monclient: found mon.a 2026-02-25T15:01:40.766 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7abcff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:40.766 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7abcff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:40.766 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac015df70 con 0x7f7ac0157780 2026-02-25T15:01:40.766 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7abcff9640 10 monclient: _finish_auth 0 2026-02-25T15:01:40.766 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7abcff9640 10 monclient: _check_auth_tickets 2026-02-25T15:01:40.766 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7abcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:40.766 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7ac675c640 5 monclient: authenticate success, global_id 4362 2026-02-25T15:01:40.766 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7ac675c640 10 monclient: _renew_subs 2026-02-25T15:01:40.766 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7ac675c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:40.767 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7ac675c640 1 -- 10.20.193.90:0/3238999154 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac015e250 con 0x7f7ac0157780 2026-02-25T15:01:40.767 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7ac675c640 10 client.4362.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:40.767 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7ac675c640 10 monclient: _renew_subs 2026-02-25T15:01:40.767 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7ac675c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:40.767 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7ac675c640 1 -- 10.20.193.90:0/3238999154 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7ac015d3e0 con 0x7f7ac0157780 2026-02-25T15:01:40.767 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7ac675c640 1 librados: init done 2026-02-25T15:01:40.767 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7ac675c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:40.767 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7ac675c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:40.767 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7ac675c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:40.767 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.751+0000 7f7ac675c640 1 -- 10.20.193.90:0/3238999154 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ac015ea20 con 0x7f7ac0157780 2026-02-25T15:01:40.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.842+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7aac0034a0 con 0x7f7ac0157780 2026-02-25T15:01:40.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.842+0000 7f7abcff9640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:40.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.842+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7aac003b50 con 0x7f7ac0157780 2026-02-25T15:01:40.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.842+0000 7f7abcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:40.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.842+0000 7f7abcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:40.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.842+0000 7f7abcff9640 10 monclient: dump: 2026-02-25T15:01:40.844 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:40.844 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:40.844 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:40.845 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:40.845 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:40.845 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:40.845 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:40.845 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:40.845 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:40.845 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:40.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.842+0000 7f7abe7fc640 4 set_mon_vals no callback set 2026-02-25T15:01:40.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.843+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f7aac003d30 con 0x7f7ac0157780 2026-02-25T15:01:40.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.843+0000 7f7abf7fe640 10 monclient: get_auth_request con 0x7f7a90079f50 auth_method 0 2026-02-25T15:01:40.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.844+0000 7f7abcff9640 10 client.4362.objecter ms_handle_connect 0x7f7a90079f50 2026-02-25T15:01:40.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.846+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f7aac095ca0 con 0x7f7ac0157780 2026-02-25T15:01:40.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.846+0000 7f7abcff9640 10 client.4362.objecter ms_dispatch2 0x7f7ac0000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:40.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.846+0000 7f7abcff9640 3 client.4362.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:40.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.846+0000 7f7abcff9640 3 client.4362.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:40.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.846+0000 7f7abcff9640 20 client.4362.objecter dump_active .. 0 homeless 2026-02-25T15:01:40.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.846+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f7aac05f600 con 0x7f7ac0157780 2026-02-25T15:01:40.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.846+0000 7f7abcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:40.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.846+0000 7f7abcff9640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:40.952 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.951+0000 7f7ac675c640 10 start_mon_command cmd=[{"prefix": "config log", "num": 1, "format": "json"}] 2026-02-25T15:01:40.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.951+0000 7f7ac675c640 10 monclient: _send_command 2 [{"prefix": "config log", "num": 1, "format": "json"}] 2026-02-25T15:01:40.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.951+0000 7f7ac675c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:40.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.951+0000 7f7ac675c640 1 -- 10.20.193.90:0/3238999154 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f7ac005e250 con 0x7f7ac0157780 2026-02-25T15:01:40.959 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.957+0000 7f7abcff9640 1 -- 10.20.193.90:0/3238999154 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v1) ==== 86+0+61 (unknown 4269106227 0 102270264) 0x7f7ac005e250 con 0x7f7ac0157780 2026-02-25T15:01:40.959 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.957+0000 7f7abcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config log", "num": 1, "format": "json"}] 2026-02-25T15:01:40.959 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.957+0000 7f7abcff9640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:40.959 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:01:40.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.959+0000 7f7ac675c640 10 librados: watch_flush enter 2026-02-25T15:01:40.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.959+0000 7f7ac675c640 10 librados: watch_flush exit 2026-02-25T15:01:40.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.959+0000 7f7ac675c640 20 client.4362.objecter shutdown clearing up homeless session... 2026-02-25T15:01:40.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.959+0000 7f7ac675c640 10 client.4362.objecter successfully canceled tick 2026-02-25T15:01:40.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.959+0000 7f7ac675c640 1 -- 10.20.193.90:0/3238999154 >> v1:10.20.193.189:6800/3647375982 conn(0x7f7a90079f50 legacy=0x7f7a9009a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:40.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.959+0000 7f7ac675c640 10 monclient: shutdown 2026-02-25T15:01:40.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.959+0000 7f7ac675c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:40.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.959+0000 7f7ac675c640 1 -- 10.20.193.90:0/3238999154 >> v1:10.20.193.90:6789/0 conn(0x7f7ac0157780 legacy=0x7f7ac015ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:40.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.959+0000 7f7ac675c640 1 -- 10.20.193.90:0/3238999154 shutdown_connections 2026-02-25T15:01:40.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.959+0000 7f7ac675c640 1 -- 10.20.193.90:0/3238999154 >> 10.20.193.90:0/3238999154 conn(0x7f7ac00b2ef0 msgr2=0x7f7ac00b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:40.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.959+0000 7f7ac675c640 1 -- 10.20.193.90:0/3238999154 shutdown_connections 2026-02-25T15:01:40.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.959+0000 7f7ac675c640 1 -- 10.20.193.90:0/3238999154 wait complete. 2026-02-25T15:01:40.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:40.959+0000 7f7ac675c640 1 librados: shutdown 2026-02-25T15:01:40.968 INFO:teuthology.orchestra.run.trial090.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-25T15:01:40.969 INFO:tasks.ceph_manager:config epoch is 1 2026-02-25T15:01:40.969 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-25T15:01:40.969 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-25T15:01:40.969 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-25T15:01:41.036 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.035+0000 7feb711c3640 10 monclient: get_monmap_and_config 2026-02-25T15:01:41.037 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.035+0000 7feb711c3640 10 monclient: build_initial_monmap 2026-02-25T15:01:41.037 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.035+0000 7feb711c3640 10 monclient: monmap: 2026-02-25T15:01:41.037 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:41.037 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.037 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:41.036364+0000 2026-02-25T15:01:41.037 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:41.036364+0000 2026-02-25T15:01:41.037 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:41.037 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.037 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:41.038 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:41.038 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:41.038 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.040 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.036+0000 7feb711c3640 1 Processor -- start 2026-02-25T15:01:41.040 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.036+0000 7feb711c3640 1 -- start start 2026-02-25T15:01:41.040 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.036+0000 7feb711c3640 10 monclient: init 2026-02-25T15:01:41.040 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.036+0000 7feb711c3640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:41.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.036+0000 7feb711c3640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:41.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.036+0000 7feb711c3640 10 monclient(hunting): picked mon.noname-b con 0x7feb6c058da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:41.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.036+0000 7feb711c3640 10 monclient(hunting): picked mon.noname-c con 0x7feb6c0b0410 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:41.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.036+0000 7feb711c3640 10 monclient(hunting): picked mon.noname-a con 0x7feb6c1626a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:41.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.036+0000 7feb711c3640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c1655c0 con 0x7feb6c1626a0 2026-02-25T15:01:41.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.036+0000 7feb711c3640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c0b12b0 con 0x7feb6c058da0 2026-02-25T15:01:41.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.036+0000 7feb711c3640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c0b1590 con 0x7feb6c0b0410 2026-02-25T15:01:41.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.036+0000 7feb711c3640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:41.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.036+0000 7feb711c3640 10 monclient(hunting): authenticate will time out at 2990.636721s 2026-02-25T15:01:41.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.037+0000 7feb6ad76640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7feb6c058da0 0x7feb6c1647a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46240/0 (socket says 10.20.193.90:46240) 2026-02-25T15:01:41.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.037+0000 7feb6ad76640 1 -- 10.20.193.90:0/3142197348 learned_addr learned my addr 10.20.193.90:0/3142197348 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:41.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 1 -- 10.20.193.90:0/3142197348 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 134969103 0 0) 0x7feb6c0b12b0 con 0x7feb6c058da0 2026-02-25T15:01:41.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 10 monclient(hunting): my global_id is 4378 2026-02-25T15:01:41.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 1 -- 10.20.193.90:0/3142197348 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb58003610 con 0x7feb6c058da0 2026-02-25T15:01:41.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 1 -- 10.20.193.90:0/3142197348 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1245779489 0 0) 0x7feb6c0b1590 con 0x7feb6c0b0410 2026-02-25T15:01:41.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.043 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.043 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 10 monclient(hunting): my global_id is 4370 2026-02-25T15:01:41.043 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 1 -- 10.20.193.90:0/3142197348 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb6c0b12b0 con 0x7feb6c0b0410 2026-02-25T15:01:41.043 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 1 -- 10.20.193.90:0/3142197348 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4226131711 0 0) 0x7feb6c1655c0 con 0x7feb6c1626a0 2026-02-25T15:01:41.043 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.043 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.043 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.043 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 10 monclient(hunting): my global_id is 4365 2026-02-25T15:01:41.043 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.038+0000 7feb69d74640 1 -- 10.20.193.90:0/3142197348 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb6c0b1590 con 0x7feb6c1626a0 2026-02-25T15:01:41.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb69d74640 1 -- 10.20.193.90:0/3142197348 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3351679404 0 0) 0x7feb6c0b12b0 con 0x7feb6c0b0410 2026-02-25T15:01:41.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb69d74640 1 -- 10.20.193.90:0/3142197348 >> v1:10.20.193.93:6789/0 conn(0x7feb6c058da0 legacy=0x7feb6c1647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb69d74640 1 -- 10.20.193.90:0/3142197348 >> v1:10.20.193.90:6789/0 conn(0x7feb6c1626a0 legacy=0x7feb6c1c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb69d74640 10 monclient: _finish_hunting 0 2026-02-25T15:01:41.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb69d74640 1 monclient: found mon.noname-c 2026-02-25T15:01:41.051 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb69d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:41.051 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb69d74640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:41.051 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb69d74640 1 -- 10.20.193.90:0/3142197348 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c0b1870 con 0x7feb6c0b0410 2026-02-25T15:01:41.051 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb69d74640 10 monclient: _finish_auth 0 2026-02-25T15:01:41.051 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb69d74640 10 monclient: _check_auth_tickets 2026-02-25T15:01:41.051 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb69d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:41.051 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb69d74640 1 -- 10.20.193.90:0/3142197348 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7feb54002df0 con 0x7feb6c0b0410 2026-02-25T15:01:41.051 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb69d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:41.051 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb69d74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:41.051 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb69d74640 10 monclient: dump: 2026-02-25T15:01:41.051 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.052 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.052 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.052 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.052 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.052 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.052 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.052 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.052 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.052 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb711c3640 5 monclient: authenticate success, global_id 4370 2026-02-25T15:01:41.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.040+0000 7feb711c3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:41.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.041+0000 7feb69d74640 1 -- 10.20.193.90:0/3142197348 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7feb54002fc0 con 0x7feb6c0b0410 2026-02-25T15:01:41.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.041+0000 7feb69d74640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:41.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.041+0000 7feb69d74640 1 -- 10.20.193.90:0/3142197348 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7feb54003400 con 0x7feb6c0b0410 2026-02-25T15:01:41.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.041+0000 7feb69d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:41.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.041+0000 7feb69d74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:41.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.041+0000 7feb69d74640 10 monclient: dump: 2026-02-25T15:01:41.053 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.053 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.053 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.054 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.054 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.054 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.054 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.054 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.054 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.054 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.054 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.041+0000 7feb711c3640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:41.054 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.041+0000 7feb711c3640 4 set_mon_vals no callback set 2026-02-25T15:01:41.054 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.041+0000 7feb711c3640 10 monclient: shutdown 2026-02-25T15:01:41.054 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.041+0000 7feb711c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:41.054 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.041+0000 7feb711c3640 1 -- 10.20.193.90:0/3142197348 >> v1:10.20.193.94:6789/0 conn(0x7feb6c0b0410 legacy=0x7feb6c164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.054 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.041+0000 7feb711c3640 1 -- 10.20.193.90:0/3142197348 shutdown_connections 2026-02-25T15:01:41.055 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.041+0000 7feb711c3640 1 -- 10.20.193.90:0/3142197348 >> 10.20.193.90:0/3142197348 conn(0x7feb6c0b65f0 msgr2=0x7feb6c0b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:41.055 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.041+0000 7feb711c3640 1 -- 10.20.193.90:0/3142197348 shutdown_connections 2026-02-25T15:01:41.055 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.041+0000 7feb711c3640 1 -- 10.20.193.90:0/3142197348 wait complete. 2026-02-25T15:01:41.055 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 10 monclient: build_initial_monmap 2026-02-25T15:01:41.055 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 10 monclient: monmap: 2026-02-25T15:01:41.055 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:41.055 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:41.055 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:41.055 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:41.055 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:41.055 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:41.055 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:41.056 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:41.056 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:41.056 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 1 librados: starting msgr at 2026-02-25T15:01:41.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 1 librados: starting objecter 2026-02-25T15:01:41.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:41.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 1 Processor -- start 2026-02-25T15:01:41.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 1 -- start start 2026-02-25T15:01:41.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 1 librados: setting wanted keys 2026-02-25T15:01:41.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 1 librados: calling monclient init 2026-02-25T15:01:41.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 10 monclient: init 2026-02-25T15:01:41.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:41.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:41.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 10 monclient(hunting): picked mon.noname-a con 0x7feb6c058da0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:41.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 10 monclient(hunting): picked mon.noname-b con 0x7feb6c0b0410 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:41.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 10 monclient(hunting): picked mon.noname-c con 0x7feb6c1626a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:41.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c1fca80 con 0x7feb6c058da0 2026-02-25T15:01:41.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c1fcd60 con 0x7feb6c0b0410 2026-02-25T15:01:41.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c1fd040 con 0x7feb6c1626a0 2026-02-25T15:01:41.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:41.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.042+0000 7feb711c3640 10 monclient(hunting): authenticate will time out at 2990.642769s 2026-02-25T15:01:41.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.043+0000 7feb6a575640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7feb6c0b0410 0x7feb6c135a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46242/0 (socket says 10.20.193.90:46242) 2026-02-25T15:01:41.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.043+0000 7feb6a575640 1 -- 10.20.193.90:0/1054578478 learned_addr learned my addr 10.20.193.90:0/1054578478 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:41.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.043+0000 7feb4b7fe640 10 client.?.objecter ms_handle_connect 0x7feb6c058da0 2026-02-25T15:01:41.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.043+0000 7feb4b7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:41.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.043+0000 7feb4b7fe640 10 client.?.objecter ms_handle_connect 0x7feb6c0b0410 2026-02-25T15:01:41.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.043+0000 7feb4b7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:41.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.043+0000 7feb4b7fe640 10 client.?.objecter ms_handle_connect 0x7feb6c1626a0 2026-02-25T15:01:41.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.043+0000 7feb4b7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:41.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3129961169 0 0) 0x7feb6c1fca80 con 0x7feb6c058da0 2026-02-25T15:01:41.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 10 monclient(hunting): my global_id is 4368 2026-02-25T15:01:41.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb38003610 con 0x7feb6c058da0 2026-02-25T15:01:41.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1879527996 0 0) 0x7feb6c1fcd60 con 0x7feb6c0b0410 2026-02-25T15:01:41.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 10 monclient(hunting): my global_id is 4381 2026-02-25T15:01:41.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb6c1fca80 con 0x7feb6c0b0410 2026-02-25T15:01:41.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1985871692 0 0) 0x7feb6c1fd040 con 0x7feb6c1626a0 2026-02-25T15:01:41.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 10 monclient(hunting): my global_id is 4373 2026-02-25T15:01:41.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.044+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb6c1fcd60 con 0x7feb6c1626a0 2026-02-25T15:01:41.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 306795364 0 0) 0x7feb6c1fcd60 con 0x7feb6c1626a0 2026-02-25T15:01:41.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7feb6c1fd040 con 0x7feb6c1626a0 2026-02-25T15:01:41.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7feb5c003330 con 0x7feb6c1626a0 2026-02-25T15:01:41.060 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:41.060 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:41.060 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:41.060 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.060 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.060 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.060 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.060 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.060 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.060 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.060 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.060 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.061 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:41.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3864271480 0 0) 0x7feb38003610 con 0x7feb6c058da0 2026-02-25T15:01:41.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7feb6c1fcd60 con 0x7feb6c058da0 2026-02-25T15:01:41.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7feb600030b0 con 0x7feb6c058da0 2026-02-25T15:01:41.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:41.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:41.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:41.061 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.061 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.061 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.061 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.061 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.062 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.062 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.062 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.062 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.062 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.062 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2911389413 0 0) 0x7feb6c1fca80 con 0x7feb6c0b0410 2026-02-25T15:01:41.062 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7feb38003610 con 0x7feb6c0b0410 2026-02-25T15:01:41.062 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7feb540033d0 con 0x7feb6c0b0410 2026-02-25T15:01:41.062 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:41.062 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:41.062 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.046+0000 7feb4b7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:41.062 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.063 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.063 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.063 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.063 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.063 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.063 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.063 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.063 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.063 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2528930599 0 0) 0x7feb6c1fd040 con 0x7feb6c1626a0 2026-02-25T15:01:41.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 >> v1:10.20.193.93:6789/0 conn(0x7feb6c0b0410 legacy=0x7feb6c135a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 >> v1:10.20.193.90:6789/0 conn(0x7feb6c058da0 legacy=0x7feb6c1fc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb4b7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:41.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb4b7fe640 1 monclient: found mon.c 2026-02-25T15:01:41.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb4b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:41.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb4b7fe640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:41.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c1fd320 con 0x7feb6c1626a0 2026-02-25T15:01:41.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb4b7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:41.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb4b7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:41.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb4b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:41.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb711c3640 5 monclient: authenticate success, global_id 4373 2026-02-25T15:01:41.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb711c3640 10 monclient: _renew_subs 2026-02-25T15:01:41.065 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb711c3640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:41.065 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb711c3640 1 -- 10.20.193.90:0/1054578478 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c1fd600 con 0x7feb6c1626a0 2026-02-25T15:01:41.065 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb711c3640 10 client.4373.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:41.065 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb711c3640 10 monclient: _renew_subs 2026-02-25T15:01:41.065 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb711c3640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:41.065 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb711c3640 1 -- 10.20.193.90:0/1054578478 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7feb6c1fbb40 con 0x7feb6c1626a0 2026-02-25T15:01:41.065 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb711c3640 1 librados: init done 2026-02-25T15:01:41.065 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb711c3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:41.065 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb711c3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:41.065 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb711c3640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:41.065 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.048+0000 7feb711c3640 1 -- 10.20.193.90:0/1054578478 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c055260 con 0x7feb6c1626a0 2026-02-25T15:01:41.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.053+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7feb5c003a80 con 0x7feb6c1626a0 2026-02-25T15:01:41.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.053+0000 7feb4b7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:41.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.053+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7feb5c003dd0 con 0x7feb6c1626a0 2026-02-25T15:01:41.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.053+0000 7feb4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:41.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.053+0000 7feb4b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:41.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.053+0000 7feb69573640 4 set_mon_vals no callback set 2026-02-25T15:01:41.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.053+0000 7feb4b7fe640 10 monclient: dump: 2026-02-25T15:01:41.067 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.067 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.068 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.068 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.068 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.068 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.068 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.068 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.068 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.068 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.068 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.057+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7feb5c01d110 con 0x7feb6c1626a0 2026-02-25T15:01:41.068 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.057+0000 7feb6ad76640 10 monclient: get_auth_request con 0x7feb3807a0f0 auth_method 0 2026-02-25T15:01:41.068 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.058+0000 7feb4b7fe640 10 client.4373.objecter ms_handle_connect 0x7feb3807a0f0 2026-02-25T15:01:41.068 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.058+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7feb5c094b50 con 0x7feb6c1626a0 2026-02-25T15:01:41.069 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.058+0000 7feb4b7fe640 10 client.4373.objecter ms_dispatch2 0x7feb6c000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:41.069 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.058+0000 7feb4b7fe640 3 client.4373.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:41.069 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.058+0000 7feb4b7fe640 3 client.4373.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:41.069 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.058+0000 7feb4b7fe640 20 client.4373.objecter dump_active .. 0 homeless 2026-02-25T15:01:41.148 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.147+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7feb6c055260 con 0x7feb6c1626a0 2026-02-25T15:01:41.148 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.147+0000 7feb4b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:41.148 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.147+0000 7feb4b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:41.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.286+0000 7feb711c3640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2026-02-25T15:01:41.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.286+0000 7feb711c3640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2026-02-25T15:01:41.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.286+0000 7feb711c3640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:41.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.286+0000 7feb711c3640 1 -- 10.20.193.90:0/1054578478 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7feb6c165ad0 con 0x7feb6c1626a0 2026-02-25T15:01:41.333 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.332+0000 7feb4b7fe640 1 -- 10.20.193.90:0/1054578478 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+196446 (unknown 481731318 0 75670187) 0x7feb5c05e4b0 con 0x7feb6c1626a0 2026-02-25T15:01:41.333 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.332+0000 7feb4b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2026-02-25T15:01:41.333 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.332+0000 7feb4b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:41.337 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:01:41.340 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.334+0000 7feb711c3640 10 librados: watch_flush enter 2026-02-25T15:01:41.340 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.334+0000 7feb711c3640 10 librados: watch_flush exit 2026-02-25T15:01:41.340 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.334+0000 7feb711c3640 20 client.4373.objecter shutdown clearing up homeless session... 2026-02-25T15:01:41.340 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.334+0000 7feb711c3640 10 client.4373.objecter successfully canceled tick 2026-02-25T15:01:41.340 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.334+0000 7feb711c3640 1 -- 10.20.193.90:0/1054578478 >> v1:10.20.193.189:6800/3647375982 conn(0x7feb3807a0f0 legacy=0x7feb3809a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.341 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.334+0000 7feb711c3640 10 monclient: shutdown 2026-02-25T15:01:41.341 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.334+0000 7feb711c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:41.341 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.334+0000 7feb711c3640 1 -- 10.20.193.90:0/1054578478 >> v1:10.20.193.94:6789/0 conn(0x7feb6c1626a0 legacy=0x7feb6c1616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.341 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.334+0000 7feb711c3640 1 -- 10.20.193.90:0/1054578478 shutdown_connections 2026-02-25T15:01:41.341 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.334+0000 7feb711c3640 1 -- 10.20.193.90:0/1054578478 >> 10.20.193.90:0/1054578478 conn(0x7feb6c0b65f0 msgr2=0x7feb6c059bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:41.341 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.334+0000 7feb711c3640 1 -- 10.20.193.90:0/1054578478 shutdown_connections 2026-02-25T15:01:41.341 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.334+0000 7feb711c3640 1 -- 10.20.193.90:0/1054578478 wait complete. 2026-02-25T15:01:41.341 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.334+0000 7feb711c3640 1 librados: shutdown 2026-02-25T15:01:41.344 INFO:teuthology.orchestra.run.trial090.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6800","nonce":3647375982}]},"active_addr":"10.20.193.189:6800/3647375982","active_change":"2026-02-25T15:00:58.971199+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4544132024016699391,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self certificates generated and signed by cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.5","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:ceph20-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:ceph20-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"how frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self certificates generated and signed by cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.5","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:ceph20-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:ceph20-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"how frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.189:0","nonce":2427347638}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.189:0","nonce":2076512995}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.189:0","nonce":745150136}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.189:0","nonce":2955001784}]}]} 2026-02-25T15:01:41.352 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-25T15:01:41.352 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-25T15:01:41.352 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T15:01:41.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.450+0000 7fd4833e1640 10 monclient: get_monmap_and_config 2026-02-25T15:01:41.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.450+0000 7fd4833e1640 10 monclient: build_initial_monmap 2026-02-25T15:01:41.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.450+0000 7fd4833e1640 10 monclient: monmap: 2026-02-25T15:01:41.452 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:41.452 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.452 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:41.451558+0000 2026-02-25T15:01:41.453 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:41.451558+0000 2026-02-25T15:01:41.453 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:41.453 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.453 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:41.453 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:41.453 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:41.453 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.451+0000 7fd4833e1640 1 Processor -- start 2026-02-25T15:01:41.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.451+0000 7fd4833e1640 1 -- start start 2026-02-25T15:01:41.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.451+0000 7fd4833e1640 10 monclient: init 2026-02-25T15:01:41.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.451+0000 7fd4833e1640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:41.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.451+0000 7fd4833e1640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:41.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.451+0000 7fd4833e1640 10 monclient(hunting): picked mon.noname-c con 0x7fd47c1603a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:41.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.451+0000 7fd4833e1640 10 monclient(hunting): picked mon.noname-b con 0x7fd47c056690 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:41.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.451+0000 7fd4833e1640 10 monclient(hunting): picked mon.noname-a con 0x7fd47c1613d0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:41.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.451+0000 7fd4833e1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd47c161850 con 0x7fd47c1613d0 2026-02-25T15:01:41.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.451+0000 7fd4833e1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd47c0592d0 con 0x7fd47c056690 2026-02-25T15:01:41.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.451+0000 7fd4833e1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd47c0595b0 con 0x7fd47c1603a0 2026-02-25T15:01:41.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.452+0000 7fd4833e1640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:41.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.452+0000 7fd4833e1640 10 monclient(hunting): authenticate will time out at 2991.052007s 2026-02-25T15:01:41.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.452+0000 7fd481957640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fd47c1613d0 0x7fd47c0b20c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44658/0 (socket says 10.20.193.90:44658) 2026-02-25T15:01:41.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.452+0000 7fd481957640 1 -- 10.20.193.90:0/4285435608 learned_addr learned my addr 10.20.193.90:0/4285435608 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:41.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.453+0000 7fd46bfff640 1 -- 10.20.193.90:0/4285435608 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 949879999 0 0) 0x7fd478002c90 con 0x7fd47c1613d0 2026-02-25T15:01:41.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.453+0000 7fd46bfff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.453+0000 7fd46bfff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.453+0000 7fd46bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.453+0000 7fd46bfff640 10 monclient(hunting): my global_id is 4371 2026-02-25T15:01:41.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.453+0000 7fd46bfff640 1 -- 10.20.193.90:0/4285435608 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd464003610 con 0x7fd47c1613d0 2026-02-25T15:01:41.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.453+0000 7fd46bfff640 1 -- 10.20.193.90:0/4285435608 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 82381252 0 0) 0x7fd47c0595b0 con 0x7fd47c1603a0 2026-02-25T15:01:41.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.454+0000 7fd46bfff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.454+0000 7fd46bfff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.454+0000 7fd46bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.454+0000 7fd46bfff640 10 monclient(hunting): my global_id is 4376 2026-02-25T15:01:41.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.454+0000 7fd46bfff640 1 -- 10.20.193.90:0/4285435608 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd478002c90 con 0x7fd47c1603a0 2026-02-25T15:01:41.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.454+0000 7fd46bfff640 1 -- 10.20.193.90:0/4285435608 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2696391337 0 0) 0x7fd47c0592d0 con 0x7fd47c056690 2026-02-25T15:01:41.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.454+0000 7fd46bfff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.454+0000 7fd46bfff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.454+0000 7fd46bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.454+0000 7fd46bfff640 10 monclient(hunting): my global_id is 4384 2026-02-25T15:01:41.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.454+0000 7fd46bfff640 1 -- 10.20.193.90:0/4285435608 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd47c0595b0 con 0x7fd47c056690 2026-02-25T15:01:41.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd46bfff640 1 -- 10.20.193.90:0/4285435608 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3204512063 0 0) 0x7fd464003610 con 0x7fd47c1613d0 2026-02-25T15:01:41.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd46bfff640 1 -- 10.20.193.90:0/4285435608 >> v1:10.20.193.94:6789/0 conn(0x7fd47c1603a0 legacy=0x7fd47c160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd46bfff640 1 -- 10.20.193.90:0/4285435608 >> v1:10.20.193.93:6789/0 conn(0x7fd47c056690 legacy=0x7fd47c057f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd46bfff640 10 monclient: _finish_hunting 0 2026-02-25T15:01:41.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd46bfff640 1 monclient: found mon.noname-a 2026-02-25T15:01:41.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd46bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:41.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd46bfff640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:41.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd46bfff640 1 -- 10.20.193.90:0/4285435608 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47c059890 con 0x7fd47c1613d0 2026-02-25T15:01:41.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd46bfff640 10 monclient: _finish_auth 0 2026-02-25T15:01:41.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd46bfff640 10 monclient: _check_auth_tickets 2026-02-25T15:01:41.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd46bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:41.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd46bfff640 1 -- 10.20.193.90:0/4285435608 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd478003340 con 0x7fd47c1613d0 2026-02-25T15:01:41.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd46bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:41.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd46bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:41.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd46bfff640 10 monclient: dump: 2026-02-25T15:01:41.464 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.464 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.465 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.465 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.465 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.465 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.465 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.465 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.465 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.465 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd4833e1640 5 monclient: authenticate success, global_id 4371 2026-02-25T15:01:41.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.455+0000 7fd4833e1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:41.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.456+0000 7fd46bfff640 1 -- 10.20.193.90:0/4285435608 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd478003510 con 0x7fd47c1613d0 2026-02-25T15:01:41.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.456+0000 7fd46bfff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:41.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.456+0000 7fd46bfff640 1 -- 10.20.193.90:0/4285435608 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd478004d10 con 0x7fd47c1613d0 2026-02-25T15:01:41.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.456+0000 7fd4833e1640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:41.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.456+0000 7fd4833e1640 4 set_mon_vals no callback set 2026-02-25T15:01:41.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.456+0000 7fd46bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:41.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.456+0000 7fd46bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:41.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.456+0000 7fd46bfff640 10 monclient: dump: 2026-02-25T15:01:41.466 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.467 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.467 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.467 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.467 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.467 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.467 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.467 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.467 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.467 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 10 monclient: shutdown 2026-02-25T15:01:41.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:41.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 1 -- 10.20.193.90:0/4285435608 >> v1:10.20.193.90:6789/0 conn(0x7fd47c1613d0 legacy=0x7fd47c0b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.468 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 1 -- 10.20.193.90:0/4285435608 shutdown_connections 2026-02-25T15:01:41.468 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 1 -- 10.20.193.90:0/4285435608 >> 10.20.193.90:0/4285435608 conn(0x7fd47c0b65f0 msgr2=0x7fd47c0b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:41.468 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 1 -- 10.20.193.90:0/4285435608 shutdown_connections 2026-02-25T15:01:41.468 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 1 -- 10.20.193.90:0/4285435608 wait complete. 2026-02-25T15:01:41.468 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 10 monclient: build_initial_monmap 2026-02-25T15:01:41.468 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 10 monclient: monmap: 2026-02-25T15:01:41.468 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:41.468 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:41.468 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:41.468 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:41.468 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:41.468 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:41.468 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:41.469 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:41.469 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:41.469 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 1 librados: starting msgr at 2026-02-25T15:01:41.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 1 librados: starting objecter 2026-02-25T15:01:41.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:41.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 1 Processor -- start 2026-02-25T15:01:41.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 1 -- start start 2026-02-25T15:01:41.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 1 librados: setting wanted keys 2026-02-25T15:01:41.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 1 librados: calling monclient init 2026-02-25T15:01:41.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.457+0000 7fd4833e1640 10 monclient: init 2026-02-25T15:01:41.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd4833e1640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:41.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd4833e1640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:41.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd4833e1640 10 monclient(hunting): picked mon.noname-b con 0x7fd47c056690 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:41.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd4833e1640 10 monclient(hunting): picked mon.noname-a con 0x7fd47c1603a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:41.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd4833e1640 10 monclient(hunting): picked mon.noname-c con 0x7fd47c1613d0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:41.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd4833e1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd47c1fe7d0 con 0x7fd47c1603a0 2026-02-25T15:01:41.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd4833e1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd47c0b2780 con 0x7fd47c056690 2026-02-25T15:01:41.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd4833e1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd47c1fdc50 con 0x7fd47c1613d0 2026-02-25T15:01:41.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd4833e1640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:41.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd4833e1640 10 monclient(hunting): authenticate will time out at 2991.058320s 2026-02-25T15:01:41.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd480955640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fd47c1603a0 0x7fd47c1fb490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44668/0 (socket says 10.20.193.90:44668) 2026-02-25T15:01:41.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd480955640 1 -- 10.20.193.90:0/2757509753 learned_addr learned my addr 10.20.193.90:0/2757509753 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:41.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd469ffb640 10 client.?.objecter ms_handle_connect 0x7fd47c1603a0 2026-02-25T15:01:41.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd469ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:41.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd469ffb640 10 client.?.objecter ms_handle_connect 0x7fd47c056690 2026-02-25T15:01:41.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.458+0000 7fd469ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:41.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.459+0000 7fd469ffb640 10 client.?.objecter ms_handle_connect 0x7fd47c1613d0 2026-02-25T15:01:41.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.459+0000 7fd469ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:41.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.459+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1287400099 0 0) 0x7fd47c1fe7d0 con 0x7fd47c1603a0 2026-02-25T15:01:41.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.459+0000 7fd469ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.459+0000 7fd469ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.459+0000 7fd469ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.459+0000 7fd469ffb640 10 monclient(hunting): my global_id is 4374 2026-02-25T15:01:41.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.459+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd44c003610 con 0x7fd47c1603a0 2026-02-25T15:01:41.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.460+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1950890770 0 0) 0x7fd47c0b2780 con 0x7fd47c056690 2026-02-25T15:01:41.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.460+0000 7fd469ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.460+0000 7fd469ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.460+0000 7fd469ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.460+0000 7fd469ffb640 10 monclient(hunting): my global_id is 4387 2026-02-25T15:01:41.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.460+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd47c1fe7d0 con 0x7fd47c056690 2026-02-25T15:01:41.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.460+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1199066007 0 0) 0x7fd47c1fdc50 con 0x7fd47c1613d0 2026-02-25T15:01:41.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.460+0000 7fd469ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.460+0000 7fd469ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.460+0000 7fd469ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.460+0000 7fd469ffb640 10 monclient(hunting): my global_id is 4379 2026-02-25T15:01:41.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.460+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd47c0b2780 con 0x7fd47c1613d0 2026-02-25T15:01:41.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2871792287 0 0) 0x7fd44c003610 con 0x7fd47c1603a0 2026-02-25T15:01:41.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd47c1fdc50 con 0x7fd47c1603a0 2026-02-25T15:01:41.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd4700030c0 con 0x7fd47c1603a0 2026-02-25T15:01:41.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:41.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:41.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 10 monclient(hunting): dump: 2026-02-25T15:01:41.476 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.476 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.476 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.476 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.476 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.476 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.476 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.476 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.476 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.476 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:41.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 462914036 0 0) 0x7fd47c0b2780 con 0x7fd47c1613d0 2026-02-25T15:01:41.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd44c003610 con 0x7fd47c1613d0 2026-02-25T15:01:41.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3538643268 0 0) 0x7fd47c1fe7d0 con 0x7fd47c056690 2026-02-25T15:01:41.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd47c0b2780 con 0x7fd47c056690 2026-02-25T15:01:41.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd46c003030 con 0x7fd47c056690 2026-02-25T15:01:41.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:41.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:41.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 10 monclient(hunting): dump: 2026-02-25T15:01:41.478 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.478 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.478 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.478 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.478 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.478 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.478 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.478 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.478 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.478 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd4780030b0 con 0x7fd47c1613d0 2026-02-25T15:01:41.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:41.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:41.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.462+0000 7fd469ffb640 10 monclient(hunting): dump: 2026-02-25T15:01:41.479 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.479 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.479 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.479 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.479 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.479 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.479 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.479 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.479 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.479 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3047381037 0 0) 0x7fd47c1fdc50 con 0x7fd47c1603a0 2026-02-25T15:01:41.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 >> v1:10.20.193.94:6789/0 conn(0x7fd47c1613d0 legacy=0x7fd47c1fbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 >> v1:10.20.193.93:6789/0 conn(0x7fd47c056690 legacy=0x7fd47c0b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd469ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:41.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd469ffb640 1 monclient: found mon.a 2026-02-25T15:01:41.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd469ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:41.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd469ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:41.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47c1fdf30 con 0x7fd47c1603a0 2026-02-25T15:01:41.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd469ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:41.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd469ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:41.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd469ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:41.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd4833e1640 5 monclient: authenticate success, global_id 4374 2026-02-25T15:01:41.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd4833e1640 10 monclient: _renew_subs 2026-02-25T15:01:41.481 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd4833e1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:41.481 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd4833e1640 1 -- 10.20.193.90:0/2757509753 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd47c1fe210 con 0x7fd47c1603a0 2026-02-25T15:01:41.481 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd4833e1640 10 client.4374.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:41.481 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd4833e1640 10 monclient: _renew_subs 2026-02-25T15:01:41.481 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd4833e1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:41.481 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd4833e1640 1 -- 10.20.193.90:0/2757509753 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd47c1f94a0 con 0x7fd47c1603a0 2026-02-25T15:01:41.481 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd4833e1640 1 librados: init done 2026-02-25T15:01:41.481 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd4833e1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:41.481 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd4833e1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:41.481 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd4833e1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:41.481 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.464+0000 7fd4833e1640 1 -- 10.20.193.90:0/2757509753 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd47c163240 con 0x7fd47c1603a0 2026-02-25T15:01:41.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.558+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd470003700 con 0x7fd47c1603a0 2026-02-25T15:01:41.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.558+0000 7fd469ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:41.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.558+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd470003ad0 con 0x7fd47c1603a0 2026-02-25T15:01:41.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.558+0000 7fd469ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:41.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.558+0000 7fd469ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:41.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.558+0000 7fd46b7fe640 4 set_mon_vals no callback set 2026-02-25T15:01:41.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.558+0000 7fd469ffb640 10 monclient: dump: 2026-02-25T15:01:41.560 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.560 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.560 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.560 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.560 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.561 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.561 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.561 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.561 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.561 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.558+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fd470003ad0 con 0x7fd47c1603a0 2026-02-25T15:01:41.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.559+0000 7fd481156640 10 monclient: get_auth_request con 0x7fd44c07a140 auth_method 0 2026-02-25T15:01:41.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.559+0000 7fd469ffb640 10 client.4374.objecter ms_handle_connect 0x7fd44c07a140 2026-02-25T15:01:41.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.562+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7fd4700959b0 con 0x7fd47c1603a0 2026-02-25T15:01:41.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.562+0000 7fd469ffb640 10 client.4374.objecter ms_dispatch2 0x7fd47c000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:41.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.562+0000 7fd469ffb640 3 client.4374.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:41.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.562+0000 7fd469ffb640 3 client.4374.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:41.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.562+0000 7fd469ffb640 20 client.4374.objecter dump_active .. 0 homeless 2026-02-25T15:01:41.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.562+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fd47005f310 con 0x7fd47c1603a0 2026-02-25T15:01:41.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.562+0000 7fd469ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:41.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.562+0000 7fd469ffb640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:41.671 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.669+0000 7fd4833e1640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:41.671 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.669+0000 7fd4833e1640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:41.671 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.669+0000 7fd4833e1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:41.671 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.669+0000 7fd4833e1640 1 -- 10.20.193.90:0/2757509753 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd47c15fa60 con 0x7fd47c1603a0 2026-02-25T15:01:41.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.685+0000 7fd469ffb640 1 -- 10.20.193.90:0/2757509753 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+20920 (unknown 634195324 0 3184408046) 0x7fd47005ec10 con 0x7fd47c1603a0 2026-02-25T15:01:41.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.685+0000 7fd469ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:41.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.685+0000 7fd469ffb640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:41.687 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:01:41.687 INFO:teuthology.orchestra.run.trial090.stdout:{"epoch":28,"fsid":"63c679b5-d219-4797-8adb-7340f3fd6819","created":"2026-02-25T15:00:56.256818+0000","modified":"2026-02-25T15:01:40.483058+0000","last_up_change":"2026-02-25T15:01:29.405841+0000","last_in_change":"2026-02-25T15:01:02.711189+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T15:01:29.228894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-25T15:01:37.037528+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":5.9899997711181641,"score_stable":5.9899997711181641,"optimal_score":0.68999999761581421,"raw_score_acting":4.130000114440918,"raw_score_stable":4.130000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ad97006e-f5c3-4e36-97a0-7fa2b250344d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6808","nonce":3221464204}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6809","nonce":3221464204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6811","nonce":3221464204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6810","nonce":3221464204}]},"public_addr":"10.20.193.90:6808/3221464204","cluster_addr":"10.20.193.90:6809/3221464204","heartbeat_back_addr":"10.20.193.90:6811/3221464204","heartbeat_front_addr":"10.20.193.90:6810/3221464204","state":["exists","up"]},{"osd":1,"uuid":"25b02f30-96b9-43a3-9691-f06a1292e36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6804","nonce":3161035071}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6805","nonce":3161035071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6807","nonce":3161035071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6806","nonce":3161035071}]},"public_addr":"10.20.193.93:6804/3161035071","cluster_addr":"10.20.193.93:6805/3161035071","heartbeat_back_addr":"10.20.193.93:6807/3161035071","heartbeat_front_addr":"10.20.193.93:6806/3161035071","state":["exists","up"]},{"osd":2,"uuid":"aeed908b-d483-498b-83ba-632e51dd05e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6800","nonce":3705139403}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6801","nonce":3705139403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6803","nonce":3705139403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6802","nonce":3705139403}]},"public_addr":"10.20.193.94:6800/3705139403","cluster_addr":"10.20.193.94:6801/3705139403","heartbeat_back_addr":"10.20.193.94:6803/3705139403","heartbeat_front_addr":"10.20.193.94:6802/3705139403","state":["exists","up"]},{"osd":3,"uuid":"c7a30346-685f-4bd2-acaa-00b98ad3284c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6813","nonce":2702305154}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6814","nonce":2702305154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6816","nonce":2702305154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6815","nonce":2702305154}]},"public_addr":"10.20.193.189:6813/2702305154","cluster_addr":"10.20.193.189:6814/2702305154","heartbeat_back_addr":"10.20.193.189:6816/2702305154","heartbeat_front_addr":"10.20.193.189:6815/2702305154","state":["exists","up"]},{"osd":4,"uuid":"d6c97301-55cc-4579-bc92-cb240c5ebe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6800","nonce":1446526134}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6802","nonce":1446526134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6805","nonce":1446526134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6803","nonce":1446526134}]},"public_addr":"10.20.193.90:6800/1446526134","cluster_addr":"10.20.193.90:6802/1446526134","heartbeat_back_addr":"10.20.193.90:6805/1446526134","heartbeat_front_addr":"10.20.193.90:6803/1446526134","state":["exists","up"]},{"osd":5,"uuid":"e3117058-2f0c-466b-a302-061fc4c120fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6800","nonce":887686167}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6801","nonce":887686167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6803","nonce":887686167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6802","nonce":887686167}]},"public_addr":"10.20.193.93:6800/887686167","cluster_addr":"10.20.193.93:6801/887686167","heartbeat_back_addr":"10.20.193.93:6803/887686167","heartbeat_front_addr":"10.20.193.93:6802/887686167","state":["exists","up"]},{"osd":6,"uuid":"f4b4a3ce-0a0b-4386-a433-dcbf65558ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6812","nonce":1114158993}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6813","nonce":1114158993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6815","nonce":1114158993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6814","nonce":1114158993}]},"public_addr":"10.20.193.94:6812/1114158993","cluster_addr":"10.20.193.94:6813/1114158993","heartbeat_back_addr":"10.20.193.94:6815/1114158993","heartbeat_front_addr":"10.20.193.94:6814/1114158993","state":["exists","up"]},{"osd":7,"uuid":"3337e549-6b6e-444e-997b-18a19beb5640","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6809","nonce":1708410075}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6810","nonce":1708410075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6812","nonce":1708410075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6811","nonce":1708410075}]},"public_addr":"10.20.193.189:6809/1708410075","cluster_addr":"10.20.193.189:6810/1708410075","heartbeat_back_addr":"10.20.193.189:6812/1708410075","heartbeat_front_addr":"10.20.193.189:6811/1708410075","state":["exists","up"]},{"osd":8,"uuid":"b292f7c4-9a41-4e48-876c-29281c5ec73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6801","nonce":2320036239}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6804","nonce":2320036239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6807","nonce":2320036239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6806","nonce":2320036239}]},"public_addr":"10.20.193.90:6801/2320036239","cluster_addr":"10.20.193.90:6804/2320036239","heartbeat_back_addr":"10.20.193.90:6807/2320036239","heartbeat_front_addr":"10.20.193.90:6806/2320036239","state":["exists","up"]},{"osd":9,"uuid":"954a7cbb-2f06-4cb9-ae68-380ac58a9f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6808","nonce":3039017972}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6809","nonce":3039017972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6811","nonce":3039017972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6810","nonce":3039017972}]},"public_addr":"10.20.193.93:6808/3039017972","cluster_addr":"10.20.193.93:6809/3039017972","heartbeat_back_addr":"10.20.193.93:6811/3039017972","heartbeat_front_addr":"10.20.193.93:6810/3039017972","state":["exists","up"]},{"osd":10,"uuid":"95ed1c74-8ca5-4317-9ef7-fc657b3cda70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6804","nonce":4185080112}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6805","nonce":4185080112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6807","nonce":4185080112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6806","nonce":4185080112}]},"public_addr":"10.20.193.94:6804/4185080112","cluster_addr":"10.20.193.94:6805/4185080112","heartbeat_back_addr":"10.20.193.94:6807/4185080112","heartbeat_front_addr":"10.20.193.94:6806/4185080112","state":["exists","up"]},{"osd":11,"uuid":"8f617b59-e8d1-42fb-8787-64e8c2117954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6801","nonce":4031575111}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6802","nonce":4031575111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6804","nonce":4031575111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6803","nonce":4031575111}]},"public_addr":"10.20.193.189:6801/4031575111","cluster_addr":"10.20.193.189:6802/4031575111","heartbeat_back_addr":"10.20.193.189:6804/4031575111","heartbeat_front_addr":"10.20.193.189:6803/4031575111","state":["exists","up"]},{"osd":12,"uuid":"4bad4371-2ca7-4023-8406-0275011eda9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6812","nonce":1139796254}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6813","nonce":1139796254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6815","nonce":1139796254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6814","nonce":1139796254}]},"public_addr":"10.20.193.90:6812/1139796254","cluster_addr":"10.20.193.90:6813/1139796254","heartbeat_back_addr":"10.20.193.90:6815/1139796254","heartbeat_front_addr":"10.20.193.90:6814/1139796254","state":["exists","up"]},{"osd":13,"uuid":"5d1ca952-93a5-43f3-9780-8bc85f9ed3d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6812","nonce":1850919728}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6813","nonce":1850919728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6815","nonce":1850919728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6814","nonce":1850919728}]},"public_addr":"10.20.193.93:6812/1850919728","cluster_addr":"10.20.193.93:6813/1850919728","heartbeat_back_addr":"10.20.193.93:6815/1850919728","heartbeat_front_addr":"10.20.193.93:6814/1850919728","state":["exists","up"]},{"osd":14,"uuid":"50853d20-2756-4720-b7aa-3020ded29189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6808","nonce":887271598}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6809","nonce":887271598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6811","nonce":887271598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6810","nonce":887271598}]},"public_addr":"10.20.193.94:6808/887271598","cluster_addr":"10.20.193.94:6809/887271598","heartbeat_back_addr":"10.20.193.94:6811/887271598","heartbeat_front_addr":"10.20.193.94:6810/887271598","state":["exists","up"]},{"osd":15,"uuid":"4445eda7-fb77-4f2c-8fda-77667df22b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6805","nonce":4039858964}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6806","nonce":4039858964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6808","nonce":4039858964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6807","nonce":4039858964}]},"public_addr":"10.20.193.189:6805/4039858964","cluster_addr":"10.20.193.189:6806/4039858964","heartbeat_back_addr":"10.20.193.189:6808/4039858964","heartbeat_front_addr":"10.20.193.189:6807/4039858964","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.787596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.833159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.652364+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:27.190420+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.727824+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.707073+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.887715+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:27.132243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.999569+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.929271+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.761416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.870660+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.873627+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:27.124178+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.868032+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.940190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T15:01:41.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.687+0000 7fd4833e1640 10 librados: watch_flush enter 2026-02-25T15:01:41.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.687+0000 7fd4833e1640 10 librados: watch_flush exit 2026-02-25T15:01:41.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.687+0000 7fd4833e1640 20 client.4374.objecter shutdown clearing up homeless session... 2026-02-25T15:01:41.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.687+0000 7fd4833e1640 10 client.4374.objecter successfully canceled tick 2026-02-25T15:01:41.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.687+0000 7fd4833e1640 1 -- 10.20.193.90:0/2757509753 >> v1:10.20.193.189:6800/3647375982 conn(0x7fd44c07a140 legacy=0x7fd44c09a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.687+0000 7fd4833e1640 10 monclient: shutdown 2026-02-25T15:01:41.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.687+0000 7fd4833e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:41.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.687+0000 7fd4833e1640 1 -- 10.20.193.90:0/2757509753 >> v1:10.20.193.90:6789/0 conn(0x7fd47c1603a0 legacy=0x7fd47c1fb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.687+0000 7fd4833e1640 1 -- 10.20.193.90:0/2757509753 shutdown_connections 2026-02-25T15:01:41.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.687+0000 7fd4833e1640 1 -- 10.20.193.90:0/2757509753 >> 10.20.193.90:0/2757509753 conn(0x7fd47c0b65f0 msgr2=0x7fd47c136d70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:41.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.687+0000 7fd4833e1640 1 -- 10.20.193.90:0/2757509753 shutdown_connections 2026-02-25T15:01:41.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.687+0000 7fd4833e1640 1 -- 10.20.193.90:0/2757509753 wait complete. 2026-02-25T15:01:41.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.687+0000 7fd4833e1640 1 librados: shutdown 2026-02-25T15:01:41.698 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-25T15:01:41.698 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T15:01:41.767 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.766+0000 7f14ad3be640 10 monclient: get_monmap_and_config 2026-02-25T15:01:41.768 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.766+0000 7f14ad3be640 10 monclient: build_initial_monmap 2026-02-25T15:01:41.768 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.766+0000 7f14ad3be640 10 monclient: monmap: 2026-02-25T15:01:41.768 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:41.768 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.768 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:41.766665+0000 2026-02-25T15:01:41.768 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:41.766665+0000 2026-02-25T15:01:41.768 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:41.768 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.768 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:41.768 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:41.768 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:41.769 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.769 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.766+0000 7f14ad3be640 1 Processor -- start 2026-02-25T15:01:41.769 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.766+0000 7f14ad3be640 1 -- start start 2026-02-25T15:01:41.769 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.766+0000 7f14ad3be640 10 monclient: init 2026-02-25T15:01:41.771 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.766+0000 7f14ad3be640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:41.771 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.766+0000 7f14ad3be640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:41.771 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.766+0000 7f14ad3be640 10 monclient(hunting): picked mon.noname-b con 0x7f14a8058da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:41.771 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.766+0000 7f14ad3be640 10 monclient(hunting): picked mon.noname-c con 0x7f14a80b0410 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:41.771 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.766+0000 7f14ad3be640 10 monclient(hunting): picked mon.noname-a con 0x7f14a81626a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:41.772 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.767+0000 7f14ad3be640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a81655c0 con 0x7f14a81626a0 2026-02-25T15:01:41.772 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.767+0000 7f14ad3be640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a80b12b0 con 0x7f14a8058da0 2026-02-25T15:01:41.772 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.767+0000 7f14ad3be640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a80b1590 con 0x7f14a80b0410 2026-02-25T15:01:41.772 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.767+0000 7f14ad3be640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:41.772 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.767+0000 7f14ad3be640 10 monclient(hunting): authenticate will time out at 2991.367028s 2026-02-25T15:01:41.772 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.767+0000 7f14a6ffd640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f14a8058da0 0x7f14a81647a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46266/0 (socket says 10.20.193.90:46266) 2026-02-25T15:01:41.772 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.767+0000 7f14a6ffd640 1 -- 10.20.193.90:0/4192335858 learned_addr learned my addr 10.20.193.90:0/4192335858 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:41.772 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.768+0000 7f14a5ffb640 1 -- 10.20.193.90:0/4192335858 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1361417406 0 0) 0x7f14a81655c0 con 0x7f14a81626a0 2026-02-25T15:01:41.772 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.768+0000 7f14a5ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.772 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.768+0000 7f14a5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.772 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.768+0000 7f14a5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.768+0000 7f14a5ffb640 10 monclient(hunting): my global_id is 4377 2026-02-25T15:01:41.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.768+0000 7f14a5ffb640 1 -- 10.20.193.90:0/4192335858 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1494003610 con 0x7f14a81626a0 2026-02-25T15:01:41.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.769+0000 7f14a5ffb640 1 -- 10.20.193.90:0/4192335858 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2292331754 0 0) 0x7f14a80b1590 con 0x7f14a80b0410 2026-02-25T15:01:41.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.769+0000 7f14a5ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.769+0000 7f14a5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.769+0000 7f14a5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.769+0000 7f14a5ffb640 10 monclient(hunting): my global_id is 4382 2026-02-25T15:01:41.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.769+0000 7f14a5ffb640 1 -- 10.20.193.90:0/4192335858 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14a81655c0 con 0x7f14a80b0410 2026-02-25T15:01:41.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.769+0000 7f14a5ffb640 1 -- 10.20.193.90:0/4192335858 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 228088535 0 0) 0x7f14a80b12b0 con 0x7f14a8058da0 2026-02-25T15:01:41.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.769+0000 7f14a5ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.769+0000 7f14a5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.769+0000 7f14a5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.769+0000 7f14a5ffb640 10 monclient(hunting): my global_id is 4390 2026-02-25T15:01:41.774 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.769+0000 7f14a5ffb640 1 -- 10.20.193.90:0/4192335858 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14a80b1590 con 0x7f14a8058da0 2026-02-25T15:01:41.778 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.770+0000 7f14a5ffb640 1 -- 10.20.193.90:0/4192335858 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3429463268 0 0) 0x7f1494003610 con 0x7f14a81626a0 2026-02-25T15:01:41.778 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 1 -- 10.20.193.90:0/4192335858 >> v1:10.20.193.94:6789/0 conn(0x7f14a80b0410 legacy=0x7f14a8164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.779 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 1 -- 10.20.193.90:0/4192335858 >> v1:10.20.193.93:6789/0 conn(0x7f14a8058da0 legacy=0x7f14a81647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.779 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:41.779 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 1 monclient: found mon.noname-a 2026-02-25T15:01:41.779 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:41.779 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:41.779 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 1 -- 10.20.193.90:0/4192335858 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a80b1870 con 0x7f14a81626a0 2026-02-25T15:01:41.779 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:41.779 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:41.779 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:41.779 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 1 -- 10.20.193.90:0/4192335858 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1498003170 con 0x7f14a81626a0 2026-02-25T15:01:41.779 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:41.780 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:41.780 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 10 monclient: dump: 2026-02-25T15:01:41.780 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.780 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.780 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.780 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.780 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.780 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.780 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.780 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.780 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.780 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.781 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14ad3be640 5 monclient: authenticate success, global_id 4377 2026-02-25T15:01:41.781 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14ad3be640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:41.781 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 1 -- 10.20.193.90:0/4192335858 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1498003340 con 0x7f14a81626a0 2026-02-25T15:01:41.781 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:41.781 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 1 -- 10.20.193.90:0/4192335858 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1498003780 con 0x7f14a81626a0 2026-02-25T15:01:41.781 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:41.781 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:41.781 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14a5ffb640 10 monclient: dump: 2026-02-25T15:01:41.781 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.781 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.781 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.781 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.781 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.782 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.782 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.782 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.782 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.782 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.782 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14ad3be640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:41.782 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.771+0000 7f14ad3be640 4 set_mon_vals no callback set 2026-02-25T15:01:41.782 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.772+0000 7f14ad3be640 10 monclient: shutdown 2026-02-25T15:01:41.782 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.772+0000 7f14ad3be640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:41.782 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.772+0000 7f14ad3be640 1 -- 10.20.193.90:0/4192335858 >> v1:10.20.193.90:6789/0 conn(0x7f14a81626a0 legacy=0x7f14a81c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.782 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.772+0000 7f14ad3be640 1 -- 10.20.193.90:0/4192335858 shutdown_connections 2026-02-25T15:01:41.782 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.772+0000 7f14ad3be640 1 -- 10.20.193.90:0/4192335858 >> 10.20.193.90:0/4192335858 conn(0x7f14a80b65f0 msgr2=0x7f14a80b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:41.783 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.772+0000 7f14ad3be640 1 -- 10.20.193.90:0/4192335858 shutdown_connections 2026-02-25T15:01:41.783 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.772+0000 7f14ad3be640 1 -- 10.20.193.90:0/4192335858 wait complete. 2026-02-25T15:01:41.783 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 10 monclient: build_initial_monmap 2026-02-25T15:01:41.783 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 10 monclient: monmap: 2026-02-25T15:01:41.783 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:41.783 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:41.783 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:41.783 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:41.783 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:41.783 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:41.783 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:41.783 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:41.783 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:41.784 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.784 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 1 librados: starting msgr at 2026-02-25T15:01:41.784 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 1 librados: starting objecter 2026-02-25T15:01:41.784 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:41.784 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 1 Processor -- start 2026-02-25T15:01:41.784 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 1 -- start start 2026-02-25T15:01:41.784 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 1 librados: setting wanted keys 2026-02-25T15:01:41.784 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 1 librados: calling monclient init 2026-02-25T15:01:41.784 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 10 monclient: init 2026-02-25T15:01:41.784 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:41.784 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:41.784 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 10 monclient(hunting): picked mon.noname-a con 0x7f14a8058da0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:41.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 10 monclient(hunting): picked mon.noname-b con 0x7f14a80b0410 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:41.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 10 monclient(hunting): picked mon.noname-c con 0x7f14a81626a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:41.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a8160890 con 0x7f14a8058da0 2026-02-25T15:01:41.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a81166c0 con 0x7f14a80b0410 2026-02-25T15:01:41.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a81169a0 con 0x7f14a81626a0 2026-02-25T15:01:41.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:41.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.773+0000 7f14ad3be640 10 monclient(hunting): authenticate will time out at 2991.373948s 2026-02-25T15:01:41.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.774+0000 7f14a6ffd640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f14a8058da0 0x7f14a8132900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44688/0 (socket says 10.20.193.90:44688) 2026-02-25T15:01:41.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.774+0000 7f14a67fc640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f14a80b0410 0x7f14a8161350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46268/0 (socket says 10.20.193.90:46268) 2026-02-25T15:01:41.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.774+0000 7f14a6ffd640 1 -- 10.20.193.90:0/3054199615 learned_addr learned my addr 10.20.193.90:0/3054199615 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:41.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.774+0000 7f14877fe640 10 client.?.objecter ms_handle_connect 0x7f14a8058da0 2026-02-25T15:01:41.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.774+0000 7f14877fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:41.786 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.774+0000 7f14877fe640 10 client.?.objecter ms_handle_connect 0x7f14a80b0410 2026-02-25T15:01:41.786 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.774+0000 7f14877fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:41.786 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.774+0000 7f14877fe640 10 client.?.objecter ms_handle_connect 0x7f14a81626a0 2026-02-25T15:01:41.786 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.774+0000 7f14877fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:41.786 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.775+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2383978172 0 0) 0x7f14a8160890 con 0x7f14a8058da0 2026-02-25T15:01:41.786 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.775+0000 7f14877fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.786 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.775+0000 7f14877fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.786 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.775+0000 7f14877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.786 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.775+0000 7f14877fe640 10 monclient(hunting): my global_id is 4380 2026-02-25T15:01:41.786 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.775+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1474003610 con 0x7f14a8058da0 2026-02-25T15:01:41.786 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.775+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2429076147 0 0) 0x7f14a81166c0 con 0x7f14a80b0410 2026-02-25T15:01:41.786 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.775+0000 7f14877fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.787 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.775+0000 7f14877fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.787 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.775+0000 7f14877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.787 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.775+0000 7f14877fe640 10 monclient(hunting): my global_id is 4393 2026-02-25T15:01:41.787 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.775+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14a8160890 con 0x7f14a80b0410 2026-02-25T15:01:41.787 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.776+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2679191892 0 0) 0x7f14a81169a0 con 0x7f14a81626a0 2026-02-25T15:01:41.787 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.776+0000 7f14877fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:41.787 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.776+0000 7f14877fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:41.787 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.776+0000 7f14877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:41.787 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.776+0000 7f14877fe640 10 monclient(hunting): my global_id is 4385 2026-02-25T15:01:41.787 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.776+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14a81166c0 con 0x7f14a81626a0 2026-02-25T15:01:41.790 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 239905238 0 0) 0x7f1474003610 con 0x7f14a8058da0 2026-02-25T15:01:41.790 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f14a81169a0 con 0x7f14a8058da0 2026-02-25T15:01:41.790 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f149c003060 con 0x7f14a8058da0 2026-02-25T15:01:41.790 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:41.791 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:41.791 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 10 monclient(hunting): dump: 2026-02-25T15:01:41.791 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.791 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.791 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.791 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.791 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.791 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.791 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.791 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.791 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.791 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.792 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:41.792 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2120235298 0 0) 0x7f14a8160890 con 0x7f14a80b0410 2026-02-25T15:01:41.792 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1474003610 con 0x7f14a80b0410 2026-02-25T15:01:41.792 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f14900030b0 con 0x7f14a80b0410 2026-02-25T15:01:41.792 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:41.792 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:41.792 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 10 monclient(hunting): dump: 2026-02-25T15:01:41.792 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.792 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.792 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.792 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.792 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.792 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.793 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.793 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.793 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.793 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.793 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1581289103 0 0) 0x7f14a81166c0 con 0x7f14a81626a0 2026-02-25T15:01:41.793 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f14a8160890 con 0x7f14a81626a0 2026-02-25T15:01:41.793 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1498003690 con 0x7f14a81626a0 2026-02-25T15:01:41.793 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:41.793 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:41.793 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.778+0000 7f14877fe640 10 monclient(hunting): dump: 2026-02-25T15:01:41.793 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.793 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.794 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.794 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.794 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.794 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.794 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.794 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.794 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.794 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.794 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3429003425 0 0) 0x7f14a81169a0 con 0x7f14a8058da0 2026-02-25T15:01:41.794 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 >> v1:10.20.193.94:6789/0 conn(0x7f14a81626a0 legacy=0x7f14a81fe200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.794 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 >> v1:10.20.193.93:6789/0 conn(0x7f14a80b0410 legacy=0x7f14a8161350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:41.794 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14877fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:41.794 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14877fe640 1 monclient: found mon.a 2026-02-25T15:01:41.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:41.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14877fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:41.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a8116c80 con 0x7f14a8058da0 2026-02-25T15:01:41.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14877fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:41.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14877fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:41.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:41.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14ad3be640 5 monclient: authenticate success, global_id 4380 2026-02-25T15:01:41.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14ad3be640 10 monclient: _renew_subs 2026-02-25T15:01:41.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14ad3be640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:41.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14ad3be640 1 -- 10.20.193.90:0/3054199615 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f14a8160b50 con 0x7f14a8058da0 2026-02-25T15:01:41.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14ad3be640 10 client.4380.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:41.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14ad3be640 10 monclient: _renew_subs 2026-02-25T15:01:41.796 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14ad3be640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:41.796 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14ad3be640 1 -- 10.20.193.90:0/3054199615 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f14a81ff090 con 0x7f14a8058da0 2026-02-25T15:01:41.796 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14ad3be640 1 librados: init done 2026-02-25T15:01:41.796 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14ad3be640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:41.796 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14ad3be640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:41.796 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14ad3be640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:41.796 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.780+0000 7f14ad3be640 1 -- 10.20.193.90:0/3054199615 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a80b6e00 con 0x7f14a8058da0 2026-02-25T15:01:41.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.874+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f149c003200 con 0x7f14a8058da0 2026-02-25T15:01:41.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.874+0000 7f14877fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:41.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.874+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f149c004af0 con 0x7f14a8058da0 2026-02-25T15:01:41.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.874+0000 7f14877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:41.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.874+0000 7f14877fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:41.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.874+0000 7f14877fe640 10 monclient: dump: 2026-02-25T15:01:41.877 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:41.877 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:41.877 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.877 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:41.877 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:41.877 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:41.877 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:41.877 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:41.877 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:41.878 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:41.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.874+0000 7f14a57fa640 4 set_mon_vals no callback set 2026-02-25T15:01:41.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.875+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f149c004af0 con 0x7f14a8058da0 2026-02-25T15:01:41.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.876+0000 7f14a67fc640 10 monclient: get_auth_request con 0x7f147407a040 auth_method 0 2026-02-25T15:01:41.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.876+0000 7f14877fe640 10 client.4380.objecter ms_handle_connect 0x7f147407a040 2026-02-25T15:01:41.880 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.879+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f149c095ae0 con 0x7f14a8058da0 2026-02-25T15:01:41.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.879+0000 7f14877fe640 10 client.4380.objecter ms_dispatch2 0x7f14a8000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:41.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.879+0000 7f14877fe640 3 client.4380.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:41.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.879+0000 7f14877fe640 3 client.4380.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:41.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.879+0000 7f14877fe640 20 client.4380.objecter dump_active .. 0 homeless 2026-02-25T15:01:41.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.879+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f149c05f440 con 0x7f14a8058da0 2026-02-25T15:01:41.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.879+0000 7f14877fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:41.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.879+0000 7f14877fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:41.986 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.984+0000 7f14ad3be640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:41.986 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.984+0000 7f14ad3be640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:41.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.984+0000 7f14ad3be640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:41.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.984+0000 7f14ad3be640 1 -- 10.20.193.90:0/3054199615 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14a80b27f0 con 0x7f14a8058da0 2026-02-25T15:01:41.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.995+0000 7f14877fe640 1 -- 10.20.193.90:0/3054199615 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+20920 (unknown 634195324 0 3184408046) 0x7f149c05ed40 con 0x7f14a8058da0 2026-02-25T15:01:41.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.995+0000 7f14877fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:01:41.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.995+0000 7f14877fe640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:41.997 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:01:41.998 INFO:teuthology.orchestra.run.trial090.stdout:{"epoch":28,"fsid":"63c679b5-d219-4797-8adb-7340f3fd6819","created":"2026-02-25T15:00:56.256818+0000","modified":"2026-02-25T15:01:40.483058+0000","last_up_change":"2026-02-25T15:01:29.405841+0000","last_in_change":"2026-02-25T15:01:02.711189+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T15:01:29.228894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-25T15:01:37.037528+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":5.9899997711181641,"score_stable":5.9899997711181641,"optimal_score":0.68999999761581421,"raw_score_acting":4.130000114440918,"raw_score_stable":4.130000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ad97006e-f5c3-4e36-97a0-7fa2b250344d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6808","nonce":3221464204}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6809","nonce":3221464204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6811","nonce":3221464204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6810","nonce":3221464204}]},"public_addr":"10.20.193.90:6808/3221464204","cluster_addr":"10.20.193.90:6809/3221464204","heartbeat_back_addr":"10.20.193.90:6811/3221464204","heartbeat_front_addr":"10.20.193.90:6810/3221464204","state":["exists","up"]},{"osd":1,"uuid":"25b02f30-96b9-43a3-9691-f06a1292e36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6804","nonce":3161035071}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6805","nonce":3161035071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6807","nonce":3161035071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6806","nonce":3161035071}]},"public_addr":"10.20.193.93:6804/3161035071","cluster_addr":"10.20.193.93:6805/3161035071","heartbeat_back_addr":"10.20.193.93:6807/3161035071","heartbeat_front_addr":"10.20.193.93:6806/3161035071","state":["exists","up"]},{"osd":2,"uuid":"aeed908b-d483-498b-83ba-632e51dd05e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6800","nonce":3705139403}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6801","nonce":3705139403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6803","nonce":3705139403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6802","nonce":3705139403}]},"public_addr":"10.20.193.94:6800/3705139403","cluster_addr":"10.20.193.94:6801/3705139403","heartbeat_back_addr":"10.20.193.94:6803/3705139403","heartbeat_front_addr":"10.20.193.94:6802/3705139403","state":["exists","up"]},{"osd":3,"uuid":"c7a30346-685f-4bd2-acaa-00b98ad3284c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6813","nonce":2702305154}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6814","nonce":2702305154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6816","nonce":2702305154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6815","nonce":2702305154}]},"public_addr":"10.20.193.189:6813/2702305154","cluster_addr":"10.20.193.189:6814/2702305154","heartbeat_back_addr":"10.20.193.189:6816/2702305154","heartbeat_front_addr":"10.20.193.189:6815/2702305154","state":["exists","up"]},{"osd":4,"uuid":"d6c97301-55cc-4579-bc92-cb240c5ebe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6800","nonce":1446526134}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6802","nonce":1446526134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6805","nonce":1446526134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6803","nonce":1446526134}]},"public_addr":"10.20.193.90:6800/1446526134","cluster_addr":"10.20.193.90:6802/1446526134","heartbeat_back_addr":"10.20.193.90:6805/1446526134","heartbeat_front_addr":"10.20.193.90:6803/1446526134","state":["exists","up"]},{"osd":5,"uuid":"e3117058-2f0c-466b-a302-061fc4c120fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6800","nonce":887686167}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6801","nonce":887686167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6803","nonce":887686167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6802","nonce":887686167}]},"public_addr":"10.20.193.93:6800/887686167","cluster_addr":"10.20.193.93:6801/887686167","heartbeat_back_addr":"10.20.193.93:6803/887686167","heartbeat_front_addr":"10.20.193.93:6802/887686167","state":["exists","up"]},{"osd":6,"uuid":"f4b4a3ce-0a0b-4386-a433-dcbf65558ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6812","nonce":1114158993}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6813","nonce":1114158993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6815","nonce":1114158993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6814","nonce":1114158993}]},"public_addr":"10.20.193.94:6812/1114158993","cluster_addr":"10.20.193.94:6813/1114158993","heartbeat_back_addr":"10.20.193.94:6815/1114158993","heartbeat_front_addr":"10.20.193.94:6814/1114158993","state":["exists","up"]},{"osd":7,"uuid":"3337e549-6b6e-444e-997b-18a19beb5640","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6809","nonce":1708410075}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6810","nonce":1708410075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6812","nonce":1708410075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6811","nonce":1708410075}]},"public_addr":"10.20.193.189:6809/1708410075","cluster_addr":"10.20.193.189:6810/1708410075","heartbeat_back_addr":"10.20.193.189:6812/1708410075","heartbeat_front_addr":"10.20.193.189:6811/1708410075","state":["exists","up"]},{"osd":8,"uuid":"b292f7c4-9a41-4e48-876c-29281c5ec73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6801","nonce":2320036239}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6804","nonce":2320036239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6807","nonce":2320036239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6806","nonce":2320036239}]},"public_addr":"10.20.193.90:6801/2320036239","cluster_addr":"10.20.193.90:6804/2320036239","heartbeat_back_addr":"10.20.193.90:6807/2320036239","heartbeat_front_addr":"10.20.193.90:6806/2320036239","state":["exists","up"]},{"osd":9,"uuid":"954a7cbb-2f06-4cb9-ae68-380ac58a9f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6808","nonce":3039017972}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6809","nonce":3039017972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6811","nonce":3039017972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6810","nonce":3039017972}]},"public_addr":"10.20.193.93:6808/3039017972","cluster_addr":"10.20.193.93:6809/3039017972","heartbeat_back_addr":"10.20.193.93:6811/3039017972","heartbeat_front_addr":"10.20.193.93:6810/3039017972","state":["exists","up"]},{"osd":10,"uuid":"95ed1c74-8ca5-4317-9ef7-fc657b3cda70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6804","nonce":4185080112}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6805","nonce":4185080112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6807","nonce":4185080112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6806","nonce":4185080112}]},"public_addr":"10.20.193.94:6804/4185080112","cluster_addr":"10.20.193.94:6805/4185080112","heartbeat_back_addr":"10.20.193.94:6807/4185080112","heartbeat_front_addr":"10.20.193.94:6806/4185080112","state":["exists","up"]},{"osd":11,"uuid":"8f617b59-e8d1-42fb-8787-64e8c2117954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6801","nonce":4031575111}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6802","nonce":4031575111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6804","nonce":4031575111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6803","nonce":4031575111}]},"public_addr":"10.20.193.189:6801/4031575111","cluster_addr":"10.20.193.189:6802/4031575111","heartbeat_back_addr":"10.20.193.189:6804/4031575111","heartbeat_front_addr":"10.20.193.189:6803/4031575111","state":["exists","up"]},{"osd":12,"uuid":"4bad4371-2ca7-4023-8406-0275011eda9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6812","nonce":1139796254}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6813","nonce":1139796254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6815","nonce":1139796254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6814","nonce":1139796254}]},"public_addr":"10.20.193.90:6812/1139796254","cluster_addr":"10.20.193.90:6813/1139796254","heartbeat_back_addr":"10.20.193.90:6815/1139796254","heartbeat_front_addr":"10.20.193.90:6814/1139796254","state":["exists","up"]},{"osd":13,"uuid":"5d1ca952-93a5-43f3-9780-8bc85f9ed3d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6812","nonce":1850919728}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6813","nonce":1850919728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6815","nonce":1850919728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6814","nonce":1850919728}]},"public_addr":"10.20.193.93:6812/1850919728","cluster_addr":"10.20.193.93:6813/1850919728","heartbeat_back_addr":"10.20.193.93:6815/1850919728","heartbeat_front_addr":"10.20.193.93:6814/1850919728","state":["exists","up"]},{"osd":14,"uuid":"50853d20-2756-4720-b7aa-3020ded29189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6808","nonce":887271598}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6809","nonce":887271598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6811","nonce":887271598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6810","nonce":887271598}]},"public_addr":"10.20.193.94:6808/887271598","cluster_addr":"10.20.193.94:6809/887271598","heartbeat_back_addr":"10.20.193.94:6811/887271598","heartbeat_front_addr":"10.20.193.94:6810/887271598","state":["exists","up"]},{"osd":15,"uuid":"4445eda7-fb77-4f2c-8fda-77667df22b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6805","nonce":4039858964}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6806","nonce":4039858964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6808","nonce":4039858964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6807","nonce":4039858964}]},"public_addr":"10.20.193.189:6805/4039858964","cluster_addr":"10.20.193.189:6806/4039858964","heartbeat_back_addr":"10.20.193.189:6808/4039858964","heartbeat_front_addr":"10.20.193.189:6807/4039858964","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.787596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.833159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.652364+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:27.190420+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.727824+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.707073+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.887715+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:27.132243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.999569+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.929271+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.761416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.870660+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.873627+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:27.124178+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.868032+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.940190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T15:01:41.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.997+0000 7f14ad3be640 10 librados: watch_flush enter 2026-02-25T15:01:41.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.997+0000 7f14ad3be640 10 librados: watch_flush exit 2026-02-25T15:01:41.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.997+0000 7f14ad3be640 20 client.4380.objecter shutdown clearing up homeless session... 2026-02-25T15:01:41.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.997+0000 7f14ad3be640 10 client.4380.objecter successfully canceled tick 2026-02-25T15:01:41.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.997+0000 7f14ad3be640 1 -- 10.20.193.90:0/3054199615 >> v1:10.20.193.189:6800/3647375982 conn(0x7f147407a040 legacy=0x7f147409a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.997+0000 7f14ad3be640 10 monclient: shutdown 2026-02-25T15:01:42.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.997+0000 7f14ad3be640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.997+0000 7f14ad3be640 1 -- 10.20.193.90:0/3054199615 >> v1:10.20.193.90:6789/0 conn(0x7f14a8058da0 legacy=0x7f14a8132900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.997+0000 7f14ad3be640 1 -- 10.20.193.90:0/3054199615 shutdown_connections 2026-02-25T15:01:42.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.997+0000 7f14ad3be640 1 -- 10.20.193.90:0/3054199615 >> 10.20.193.90:0/3054199615 conn(0x7f14a80b65f0 msgr2=0x7f14a8055a80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.997+0000 7f14ad3be640 1 -- 10.20.193.90:0/3054199615 shutdown_connections 2026-02-25T15:01:42.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.997+0000 7f14ad3be640 1 -- 10.20.193.90:0/3054199615 wait complete. 2026-02-25T15:01:42.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:41.997+0000 7f14ad3be640 1 librados: shutdown 2026-02-25T15:01:42.009 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-25T15:01:42.010 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-25T15:01:42.010 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-25T15:01:42.011 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-25T15:01:42.011 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2026-02-25T15:01:42.012 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2026-02-25T15:01:42.012 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2026-02-25T15:01:42.013 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2026-02-25T15:01:42.013 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2026-02-25T15:01:42.014 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2026-02-25T15:01:42.014 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2026-02-25T15:01:42.014 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2026-02-25T15:01:42.015 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2026-02-25T15:01:42.015 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2026-02-25T15:01:42.016 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2026-02-25T15:01:42.016 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2026-02-25T15:01:42.098 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.096+0000 7f26d7c5b640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.098 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.096+0000 7f26d7c5b640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.098 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.096+0000 7f26d7c5b640 10 monclient: monmap: 2026-02-25T15:01:42.098 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.098 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.098 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.097407+0000 2026-02-25T15:01:42.098 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.097407+0000 2026-02-25T15:01:42.098 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.099 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.099 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.099 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.099 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.099 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7fc048973640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7fc048973640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7fc048973640 10 monclient: monmap: 2026-02-25T15:01:42.100 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.100 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.100 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.097872+0000 2026-02-25T15:01:42.100 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.097872+0000 2026-02-25T15:01:42.100 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.100 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.100 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.100 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.100 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.101 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7f26d7c5b640 1 Processor -- start 2026-02-25T15:01:42.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7f26d7c5b640 1 -- start start 2026-02-25T15:01:42.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7f26d7c5b640 10 monclient: init 2026-02-25T15:01:42.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7faaf52d1640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7faaf52d1640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7faaf52d1640 10 monclient: monmap: 2026-02-25T15:01:42.102 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.102 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.102 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.098255+0000 2026-02-25T15:01:42.102 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.098255+0000 2026-02-25T15:01:42.102 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.102 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.102 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.103 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.103 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.103 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7f26d7c5b640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7f26d7c5b640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:42.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7f26d7c5b640 10 monclient(hunting): picked mon.noname-b con 0x7f26d0157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7f26d7c5b640 10 monclient(hunting): picked mon.noname-c con 0x7f26d0157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7f26d7c5b640 10 monclient(hunting): picked mon.noname-a con 0x7f26d0164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7f26d7c5b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d005de70 con 0x7f26d0164d00 2026-02-25T15:01:42.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7f26d7c5b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d0055b20 con 0x7f26d0157780 2026-02-25T15:01:42.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7f26d7c5b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d0055e00 con 0x7f26d0157b50 2026-02-25T15:01:42.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7f26d7c5b640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7f26d7c5b640 10 monclient(hunting): authenticate will time out at 2991.697979s 2026-02-25T15:01:42.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7fc048973640 1 Processor -- start 2026-02-25T15:01:42.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7fc048973640 1 -- start start 2026-02-25T15:01:42.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.097+0000 7fc048973640 10 monclient: init 2026-02-25T15:01:42.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7faaf52d1640 1 Processor -- start 2026-02-25T15:01:42.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7fc048973640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7fc048973640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:42.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7fc048973640 10 monclient(hunting): picked mon.noname-b con 0x7fc040161640 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7fc048973640 10 monclient(hunting): picked mon.noname-c con 0x7fc040161a10 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7f26d59d0640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f26d0157780 0x7f26d0056ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46270/0 (socket says 10.20.193.90:46270) 2026-02-25T15:01:42.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7fc048973640 10 monclient(hunting): picked mon.noname-a con 0x7fc0400b5da0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7fc048973640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0400b8540 con 0x7fc0400b5da0 2026-02-25T15:01:42.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7f26d59d0640 1 -- 10.20.193.90:0/838923256 learned_addr learned my addr 10.20.193.90:0/838923256 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7fc048973640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0400afd90 con 0x7fc040161640 2026-02-25T15:01:42.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7fc048973640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0400b0050 con 0x7fc040161a10 2026-02-25T15:01:42.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7fc048973640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7fc048973640 10 monclient(hunting): authenticate will time out at 2991.698459s 2026-02-25T15:01:42.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7faaf52d1640 1 -- start start 2026-02-25T15:01:42.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7faaf52d1640 10 monclient: init 2026-02-25T15:01:42.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7f2a9ec55640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7f2a9ec55640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7f2a9ec55640 10 monclient: monmap: 2026-02-25T15:01:42.109 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.109 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.109 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.099479+0000 2026-02-25T15:01:42.109 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.099479+0000 2026-02-25T15:01:42.109 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.109 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.109 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.109 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.110 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.110 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7fc045ee7640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fc040161a10 0x7fc0400b7e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:53976/0 (socket says 10.20.193.90:53976) 2026-02-25T15:01:42.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.098+0000 7fc045ee7640 1 -- 10.20.193.90:0/3278426197 learned_addr learned my addr 10.20.193.90:0/3278426197 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7faaf52d1640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7faaf52d1640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:42.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7faaf52d1640 10 monclient(hunting): picked mon.noname-b con 0x7faaf0155b20 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7faaf52d1640 10 monclient(hunting): picked mon.noname-a con 0x7faaf0056c70 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7faaf52d1640 10 monclient(hunting): picked mon.noname-c con 0x7faaf0162f10 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7faaf52d1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf00b18f0 con 0x7faaf0056c70 2026-02-25T15:01:42.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7faaf52d1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf00b1bd0 con 0x7faaf0155b20 2026-02-25T15:01:42.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7faaf52d1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf00b1eb0 con 0x7faaf0162f10 2026-02-25T15:01:42.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7faaf52d1640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7faaf52d1640 10 monclient(hunting): authenticate will time out at 2991.699294s 2026-02-25T15:01:42.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7f2a9ec55640 1 Processor -- start 2026-02-25T15:01:42.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7f2a9ec55640 1 -- start start 2026-02-25T15:01:42.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7f2a9ec55640 10 monclient: init 2026-02-25T15:01:42.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7fee1f4c9640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7fee1f4c9640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.113 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7fee1f4c9640 10 monclient: monmap: 2026-02-25T15:01:42.113 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.113 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.113 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.100412+0000 2026-02-25T15:01:42.113 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.100412+0000 2026-02-25T15:01:42.113 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.113 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.113 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.113 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.113 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.113 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7f2a9ec55640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7f2a9ec55640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:42.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7f2a9ec55640 10 monclient(hunting): picked mon.noname-c con 0x7f2a980b9f80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f26d49ce640 1 -- 10.20.193.90:0/838923256 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3437909163 0 0) 0x7f26d0055e00 con 0x7f26d0157b50 2026-02-25T15:01:42.115 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f26d49ce640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.115 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f26d49ce640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.115 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.099+0000 7f2a9ec55640 10 monclient(hunting): picked mon.noname-b con 0x7f2a980ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.115 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f2a9ec55640 10 monclient(hunting): picked mon.noname-a con 0x7f2a980585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.115 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f2a9ec55640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a98163930 con 0x7f2a980585e0 2026-02-25T15:01:42.115 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f2a9ec55640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a98163bb0 con 0x7f2a980ba350 2026-02-25T15:01:42.115 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f2a9ec55640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a98163e90 con 0x7f2a980b9f80 2026-02-25T15:01:42.115 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f2a9ec55640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f2a9ec55640 10 monclient(hunting): authenticate will time out at 2991.700052s 2026-02-25T15:01:42.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f26d49ce640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7faaeeffd640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7faaf0155b20 0x7faaf00b0ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46298/0 (socket says 10.20.193.90:46298) 2026-02-25T15:01:42.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7faaeeffd640 1 -- 10.20.193.90:0/2496819441 learned_addr learned my addr 10.20.193.90:0/2496819441 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f26d49ce640 10 monclient(hunting): my global_id is 4388 2026-02-25T15:01:42.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f26d49ce640 1 -- 10.20.193.90:0/838923256 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26b8003610 con 0x7f26d0157b50 2026-02-25T15:01:42.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f26d49ce640 1 -- 10.20.193.90:0/838923256 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 32857482 0 0) 0x7f26d0055b20 con 0x7f26d0157780 2026-02-25T15:01:42.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f26d49ce640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f26d49ce640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f26d49ce640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f26d49ce640 10 monclient(hunting): my global_id is 4396 2026-02-25T15:01:42.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f26d49ce640 1 -- 10.20.193.90:0/838923256 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26d0055e00 con 0x7f26d0157780 2026-02-25T15:01:42.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7fee1f4c9640 1 Processor -- start 2026-02-25T15:01:42.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7fee1f4c9640 1 -- start start 2026-02-25T15:01:42.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7fee1f4c9640 10 monclient: init 2026-02-25T15:01:42.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f515f98a640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f2a8ffff640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f2a980ba350 0x7f2a98057ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46304/0 (socket says 10.20.193.90:46304) 2026-02-25T15:01:42.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f2a8ffff640 1 -- 10.20.193.90:0/3406671501 learned_addr learned my addr 10.20.193.90:0/3406671501 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f515f98a640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7f515f98a640 10 monclient: monmap: 2026-02-25T15:01:42.120 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.120 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.120 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.101345+0000 2026-02-25T15:01:42.120 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.101345+0000 2026-02-25T15:01:42.120 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.120 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.121 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.121 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.121 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.121 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7fee1f4c9640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7fee1f4c9640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:42.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7fee1f4c9640 10 monclient(hunting): picked mon.noname-c con 0x7fee18059e70 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7fee1f4c9640 10 monclient(hunting): picked mon.noname-b con 0x7fee18058580 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7fee1f4c9640 10 monclient(hunting): picked mon.noname-a con 0x7fee1805a240 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7fee1f4c9640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee18160560 con 0x7fee1805a240 2026-02-25T15:01:42.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7fee1f4c9640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee18160820 con 0x7fee18058580 2026-02-25T15:01:42.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7fee1f4c9640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee18160b00 con 0x7fee18059e70 2026-02-25T15:01:42.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7fee1f4c9640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.100+0000 7fee1f4c9640 10 monclient(hunting): authenticate will time out at 2991.700994s 2026-02-25T15:01:42.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f515f98a640 1 Processor -- start 2026-02-25T15:01:42.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f26d49ce640 1 -- 10.20.193.90:0/838923256 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4160891654 0 0) 0x7f26d005de70 con 0x7f26d0164d00 2026-02-25T15:01:42.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f26d49ce640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f26d49ce640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f26d49ce640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f26d49ce640 10 monclient(hunting): my global_id is 4383 2026-02-25T15:01:42.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f26d49ce640 1 -- 10.20.193.90:0/838923256 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26d0055b20 con 0x7f26d0164d00 2026-02-25T15:01:42.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f515f98a640 1 -- start start 2026-02-25T15:01:42.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f515f98a640 10 monclient: init 2026-02-25T15:01:42.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f515f98a640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f515f98a640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:42.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f515f98a640 10 monclient(hunting): picked mon.noname-c con 0x7f51580b9f80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f515f98a640 10 monclient(hunting): picked mon.noname-a con 0x7f51580ba350 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f515f98a640 10 monclient(hunting): picked mon.noname-b con 0x7f51580585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f515f98a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5158163930 con 0x7f51580ba350 2026-02-25T15:01:42.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f515f98a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5158163bb0 con 0x7f51580585e0 2026-02-25T15:01:42.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f515f98a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5158163e90 con 0x7f51580b9f80 2026-02-25T15:01:42.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f515f98a640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.101+0000 7f515f98a640 10 monclient(hunting): authenticate will time out at 2991.701960s 2026-02-25T15:01:42.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7fc0456e6640 1 -- 10.20.193.90:0/3278426197 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3282699926 0 0) 0x7fc0400b0050 con 0x7fc040161a10 2026-02-25T15:01:42.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7fc0456e6640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7fc0456e6640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7fc0456e6640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7fc0456e6640 10 monclient(hunting): my global_id is 4391 2026-02-25T15:01:42.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7fc0456e6640 1 -- 10.20.193.90:0/3278426197 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc030003610 con 0x7fc040161a10 2026-02-25T15:01:42.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7f47d1513640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7f47d1513640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7f47d1513640 10 monclient: monmap: 2026-02-25T15:01:42.127 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.128 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.128 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.104003+0000 2026-02-25T15:01:42.128 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.104003+0000 2026-02-25T15:01:42.128 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.128 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.128 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.128 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.128 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.128 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7fee1d23e640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fee18059e70 0x7fee1815fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54006/0 (socket says 10.20.193.90:54006) 2026-02-25T15:01:42.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7fee1d23e640 1 -- 10.20.193.90:0/3928160845 learned_addr learned my addr 10.20.193.90:0/3928160845 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7faaedffb640 1 -- 10.20.193.90:0/2496819441 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2297364933 0 0) 0x7faaf00b1bd0 con 0x7faaf0155b20 2026-02-25T15:01:42.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7faaedffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7faaedffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7f47d1513640 1 Processor -- start 2026-02-25T15:01:42.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7faaedffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7faaedffb640 10 monclient(hunting): my global_id is 4402 2026-02-25T15:01:42.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7faaedffb640 1 -- 10.20.193.90:0/2496819441 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faadc003610 con 0x7faaf0155b20 2026-02-25T15:01:42.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7f47d1513640 1 -- start start 2026-02-25T15:01:42.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.103+0000 7f47d1513640 10 monclient: init 2026-02-25T15:01:42.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7fc0456e6640 1 -- 10.20.193.90:0/3278426197 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3834470638 0 0) 0x7fc034002c90 con 0x7fc0400b5da0 2026-02-25T15:01:42.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7fc0456e6640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7fc0456e6640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7fc0456e6640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7fc0456e6640 10 monclient(hunting): my global_id is 4386 2026-02-25T15:01:42.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7fc0456e6640 1 -- 10.20.193.90:0/3278426197 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0400b0050 con 0x7fc0400b5da0 2026-02-25T15:01:42.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7fc0456e6640 1 -- 10.20.193.90:0/3278426197 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3873453708 0 0) 0x7fc0400afd90 con 0x7fc040161640 2026-02-25T15:01:42.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7fc0456e6640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7fc0456e6640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7fc0456e6640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7fc0456e6640 10 monclient(hunting): my global_id is 4399 2026-02-25T15:01:42.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7fc0456e6640 1 -- 10.20.193.90:0/3278426197 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc034002c90 con 0x7fc040161640 2026-02-25T15:01:42.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7f47d1513640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7f47d1513640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:42.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7f47d1513640 10 monclient(hunting): picked mon.noname-c con 0x7f47cc0b9f80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7f47d1513640 10 monclient(hunting): picked mon.noname-a con 0x7f47cc0ba350 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7f47d1513640 10 monclient(hunting): picked mon.noname-b con 0x7f47cc0585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7f47d1513640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47cc163930 con 0x7f47cc0ba350 2026-02-25T15:01:42.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7f47d1513640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47cc163bb0 con 0x7f47cc0585e0 2026-02-25T15:01:42.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7f47d1513640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47cc163e90 con 0x7f47cc0b9f80 2026-02-25T15:01:42.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7f47d1513640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.104+0000 7f47d1513640 10 monclient(hunting): authenticate will time out at 2991.704581s 2026-02-25T15:01:42.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f515df00640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f51580585e0 0x7f5158166cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46320/0 (socket says 10.20.193.90:46320) 2026-02-25T15:01:42.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f515df00640 1 -- 10.20.193.90:0/2272946097 learned_addr learned my addr 10.20.193.90:0/2272946097 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f26d49ce640 1 -- 10.20.193.90:0/838923256 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3333832857 0 0) 0x7f26b8003610 con 0x7f26d0157b50 2026-02-25T15:01:42.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f26d49ce640 1 -- 10.20.193.90:0/838923256 >> v1:10.20.193.93:6789/0 conn(0x7f26d0157780 legacy=0x7f26d0056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f26d49ce640 1 -- 10.20.193.90:0/838923256 >> v1:10.20.193.90:6789/0 conn(0x7f26d0164d00 legacy=0x7f26d01c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f26d49ce640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f26d49ce640 1 monclient: found mon.noname-c 2026-02-25T15:01:42.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f2a8f7fe640 1 -- 10.20.193.90:0/3406671501 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3324671521 0 0) 0x7f2a98163e90 con 0x7f2a980b9f80 2026-02-25T15:01:42.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f2a8f7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f2a8f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f2a8f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f2a8f7fe640 10 monclient(hunting): my global_id is 4397 2026-02-25T15:01:42.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f2a8f7fe640 1 -- 10.20.193.90:0/3406671501 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a78003610 con 0x7f2a980b9f80 2026-02-25T15:01:42.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f26d49ce640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f26d49ce640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f26d49ce640 1 -- 10.20.193.90:0/838923256 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d00560e0 con 0x7f26d0157b50 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f26d49ce640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f26d49ce640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f26d49ce640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f26d49ce640 1 -- 10.20.193.90:0/838923256 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f26c4002df0 con 0x7f26d0157b50 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.105+0000 7f26d49ce640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7f26d49ce640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7f26d49ce640 10 monclient: dump: 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.137 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.138 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7faaedffb640 1 -- 10.20.193.90:0/2496819441 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 516688479 0 0) 0x7faaf00b1eb0 con 0x7faaf0162f10 2026-02-25T15:01:42.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7faaedffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7faaedffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7faaedffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7faaedffb640 10 monclient(hunting): my global_id is 4394 2026-02-25T15:01:42.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7faaedffb640 1 -- 10.20.193.90:0/2496819441 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faaf00b1bd0 con 0x7faaf0162f10 2026-02-25T15:01:42.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7f47caffd640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f47cc0b9f80 0x7f47cc163220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54030/0 (socket says 10.20.193.90:54030) 2026-02-25T15:01:42.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7f47caffd640 1 -- 10.20.193.90:0/1900780581 learned_addr learned my addr 10.20.193.90:0/1900780581 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7f0c00599640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7f0c00599640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7f0c00599640 10 monclient: monmap: 2026-02-25T15:01:42.139 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.139 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.139 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.107183+0000 2026-02-25T15:01:42.139 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.107183+0000 2026-02-25T15:01:42.139 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.139 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.140 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.140 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.140 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.140 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7f2a8f7fe640 1 -- 10.20.193.90:0/3406671501 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4037698978 0 0) 0x7f2a98163bb0 con 0x7f2a980ba350 2026-02-25T15:01:42.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7f2a8f7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7f2a8f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7f2a8f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7f2a8f7fe640 10 monclient(hunting): my global_id is 4405 2026-02-25T15:01:42.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.106+0000 7f2a8f7fe640 1 -- 10.20.193.90:0/3406671501 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a98163e90 con 0x7f2a980ba350 2026-02-25T15:01:42.141 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.107+0000 7f26d7c5b640 5 monclient: authenticate success, global_id 4388 2026-02-25T15:01:42.141 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.107+0000 7f26d7c5b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.141 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.107+0000 7f0c00599640 1 Processor -- start 2026-02-25T15:01:42.141 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.107+0000 7f0c00599640 1 -- start start 2026-02-25T15:01:42.141 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.107+0000 7f0c00599640 10 monclient: init 2026-02-25T15:01:42.141 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f4f826b1640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f4f826b1640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f4f826b1640 10 monclient: monmap: 2026-02-25T15:01:42.142 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.142 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.142 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.108718+0000 2026-02-25T15:01:42.142 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.108718+0000 2026-02-25T15:01:42.142 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.142 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.142 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.142 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.142 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.142 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f0c00599640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f0c00599640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:42.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f0c00599640 10 monclient(hunting): picked mon.noname-a con 0x7f0bf815d0c0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f0c00599640 10 monclient(hunting): picked mon.noname-b con 0x7f0bf80b0ca0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f0c00599640 10 monclient(hunting): picked mon.noname-c con 0x7f0bf80b28b0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f0c00599640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bf80b8540 con 0x7f0bf815d0c0 2026-02-25T15:01:42.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f0c00599640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bf80b5da0 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f0c00599640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bf80b6080 con 0x7f0bf80b28b0 2026-02-25T15:01:42.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f0c00599640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f0c00599640 10 monclient(hunting): authenticate will time out at 2991.708345s 2026-02-25T15:01:42.144 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f4f826b1640 1 Processor -- start 2026-02-25T15:01:42.144 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f4f826b1640 1 -- start start 2026-02-25T15:01:42.144 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.108+0000 7f4f826b1640 10 monclient: init 2026-02-25T15:01:42.144 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7f4f826b1640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.144 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7f4f826b1640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:42.144 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7f4f826b1640 10 monclient(hunting): picked mon.noname-b con 0x7f4f7c15b220 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.144 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7f4f826b1640 10 monclient(hunting): picked mon.noname-c con 0x7f4f7c059470 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.144 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7f4f826b1640 10 monclient(hunting): picked mon.noname-a con 0x7f4f7c055f60 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.144 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7f4f826b1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7c165110 con 0x7f4f7c055f60 2026-02-25T15:01:42.144 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7f4f826b1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7c1653f0 con 0x7f4f7c15b220 2026-02-25T15:01:42.145 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7f4f826b1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7c161050 con 0x7f4f7c059470 2026-02-25T15:01:42.145 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7f4f826b1640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.145 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7f4f826b1640 10 monclient(hunting): authenticate will time out at 2991.709364s 2026-02-25T15:01:42.145 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaedffb640 1 -- 10.20.193.90:0/2496819441 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3333401539 0 0) 0x7faadc003610 con 0x7faaf0155b20 2026-02-25T15:01:42.145 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaedffb640 1 -- 10.20.193.90:0/2496819441 >> v1:10.20.193.94:6789/0 conn(0x7faaf0162f10 legacy=0x7faaf00b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.145 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaedffb640 1 -- 10.20.193.90:0/2496819441 >> v1:10.20.193.90:6789/0 conn(0x7faaf0056c70 legacy=0x7faaf0162800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.145 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaedffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaedffb640 1 monclient: found mon.noname-b 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaedffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaedffb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaedffb640 1 -- 10.20.193.90:0/2496819441 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf00b2190 con 0x7faaf0155b20 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaedffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaedffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaedffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaedffb640 1 -- 10.20.193.90:0/2496819441 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7faae4002df0 con 0x7faaf0155b20 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaedffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaedffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaedffb640 10 monclient: dump: 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.146 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.147 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.147 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaf52d1640 5 monclient: authenticate success, global_id 4402 2026-02-25T15:01:42.147 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.109+0000 7faaf52d1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.147 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc0456e6640 1 -- 10.20.193.90:0/3278426197 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2100773065 0 0) 0x7fc030003610 con 0x7fc040161a10 2026-02-25T15:01:42.147 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7f512d9b3640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.147 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7f512d9b3640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.147 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7f512d9b3640 10 monclient: monmap: 2026-02-25T15:01:42.147 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.147 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.147 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.111030+0000 2026-02-25T15:01:42.147 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.111030+0000 2026-02-25T15:01:42.147 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.147 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.147 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.147 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.148 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.148 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.148 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc0456e6640 1 -- 10.20.193.90:0/3278426197 >> v1:10.20.193.93:6789/0 conn(0x7fc040161640 legacy=0x7fc0400b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.148 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc0456e6640 1 -- 10.20.193.90:0/3278426197 >> v1:10.20.193.90:6789/0 conn(0x7fc0400b5da0 legacy=0x7fc0401ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.148 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fee07fff640 1 -- 10.20.193.90:0/3928160845 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2376158882 0 0) 0x7fee18160b00 con 0x7fee18059e70 2026-02-25T15:01:42.148 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fee07fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.148 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fee07fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.148 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fee07fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.148 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fee07fff640 10 monclient(hunting): my global_id is 4400 2026-02-25T15:01:42.148 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc0456e6640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc0456e6640 1 monclient: found mon.noname-c 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc0456e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc0456e6640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc0456e6640 1 -- 10.20.193.90:0/3278426197 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0400b0330 con 0x7fc040161a10 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc0456e6640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc0456e6640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc0456e6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc0456e6640 1 -- 10.20.193.90:0/3278426197 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fc02c002df0 con 0x7fc040161a10 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc0456e6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc0456e6640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc0456e6640 10 monclient: dump: 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.149 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fee07fff640 1 -- 10.20.193.90:0/3928160845 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee00003610 con 0x7fee18059e70 2026-02-25T15:01:42.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc048973640 5 monclient: authenticate success, global_id 4391 2026-02-25T15:01:42.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fc048973640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7f5147fff640 1 -- 10.20.193.90:0/2272946097 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 311349608 0 0) 0x7f5158163bb0 con 0x7f51580585e0 2026-02-25T15:01:42.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7f5147fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7f5147fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.151 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7f5147fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.151 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7f5147fff640 10 monclient(hunting): my global_id is 4408 2026-02-25T15:01:42.151 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7f5147fff640 1 -- 10.20.193.90:0/2272946097 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5140003610 con 0x7f51580585e0 2026-02-25T15:01:42.151 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fee07fff640 1 -- 10.20.193.90:0/3928160845 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3459040715 0 0) 0x7fee18160820 con 0x7fee18058580 2026-02-25T15:01:42.151 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fee07fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.151 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fee07fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.151 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fee07fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.151 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fee07fff640 10 monclient(hunting): my global_id is 4411 2026-02-25T15:01:42.151 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7fee07fff640 1 -- 10.20.193.90:0/3928160845 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee18160b00 con 0x7fee18058580 2026-02-25T15:01:42.151 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7f512d9b3640 1 Processor -- start 2026-02-25T15:01:42.151 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7f512d9b3640 1 -- start start 2026-02-25T15:01:42.151 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.110+0000 7f512d9b3640 10 monclient: init 2026-02-25T15:01:42.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f5147fff640 1 -- 10.20.193.90:0/2272946097 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3928408515 0 0) 0x7f514c002910 con 0x7f51580ba350 2026-02-25T15:01:42.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f5147fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f5147fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f5147fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f5147fff640 10 monclient(hunting): my global_id is 4392 2026-02-25T15:01:42.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f5147fff640 1 -- 10.20.193.90:0/2272946097 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5158163bb0 con 0x7f51580ba350 2026-02-25T15:01:42.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f5b9869f640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f5b9869f640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f5b9869f640 10 monclient: monmap: 2026-02-25T15:01:42.152 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.152 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.152 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.111966+0000 2026-02-25T15:01:42.152 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.111966+0000 2026-02-25T15:01:42.153 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.153 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.153 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.153 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.153 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.153 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f512d9b3640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f512d9b3640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:42.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f512d9b3640 10 monclient(hunting): picked mon.noname-b con 0x7f51280b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f512d9b3640 10 monclient(hunting): picked mon.noname-a con 0x7f51280ba350 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f512d9b3640 10 monclient(hunting): picked mon.noname-c con 0x7f51280585e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f512d9b3640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5128163930 con 0x7f51280ba350 2026-02-25T15:01:42.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f512d9b3640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5128163bb0 con 0x7f51280b9f80 2026-02-25T15:01:42.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f512d9b3640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5128163e90 con 0x7f51280585e0 2026-02-25T15:01:42.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f512d9b3640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f512d9b3640 10 monclient(hunting): authenticate will time out at 2991.711561s 2026-02-25T15:01:42.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f5b9869f640 1 Processor -- start 2026-02-25T15:01:42.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f5b9869f640 1 -- start start 2026-02-25T15:01:42.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f5b9869f640 10 monclient: init 2026-02-25T15:01:42.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7fee07fff640 1 -- 10.20.193.90:0/3928160845 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3697345171 0 0) 0x7fee18160560 con 0x7fee1805a240 2026-02-25T15:01:42.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7fee07fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7fee07fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7fee07fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7fee07fff640 10 monclient(hunting): my global_id is 4389 2026-02-25T15:01:42.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7fee07fff640 1 -- 10.20.193.90:0/3928160845 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee18160820 con 0x7fee1805a240 2026-02-25T15:01:42.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f73db755640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f73db755640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.111+0000 7f73db755640 10 monclient: monmap: 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.112522+0000 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.112522+0000 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f5b9869f640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f5b9869f640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f5b9869f640 10 monclient(hunting): picked mon.noname-b con 0x7f5b900b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.155 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f73db755640 1 Processor -- start 2026-02-25T15:01:42.156 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f5b9869f640 10 monclient(hunting): picked mon.noname-c con 0x7f5b900ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.156 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f5b9869f640 10 monclient(hunting): picked mon.noname-a con 0x7f5b900585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.156 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f5b9869f640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b90163930 con 0x7f5b900585e0 2026-02-25T15:01:42.156 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f73db755640 1 -- start start 2026-02-25T15:01:42.156 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f73db755640 10 monclient: init 2026-02-25T15:01:42.156 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f5b9869f640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b90163bb0 con 0x7f5b900b9f80 2026-02-25T15:01:42.156 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f5b9869f640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b90163e90 con 0x7f5b900ba350 2026-02-25T15:01:42.156 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f5b9869f640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f5b9869f640 10 monclient(hunting): authenticate will time out at 2991.712445s 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7fc84ff2e640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7fc84ff2e640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7fc84ff2e640 10 monclient: monmap: 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.113101+0000 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.113101+0000 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f47c9ffb640 1 -- 10.20.193.90:0/1900780581 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3773381627 0 0) 0x7f47cc163e90 con 0x7f47cc0b9f80 2026-02-25T15:01:42.157 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f47c9ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f47c9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f47c9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f47c9ffb640 10 monclient(hunting): my global_id is 4403 2026-02-25T15:01:42.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.112+0000 7f47c9ffb640 1 -- 10.20.193.90:0/1900780581 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47b8003610 con 0x7f47cc0b9f80 2026-02-25T15:01:42.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f73db755640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f73db755640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:42.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f73db755640 10 monclient(hunting): picked mon.noname-a con 0x7f73d40b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7fc84ff2e640 1 Processor -- start 2026-02-25T15:01:42.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f73db755640 10 monclient(hunting): picked mon.noname-b con 0x7f73d40ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f73db755640 10 monclient(hunting): picked mon.noname-c con 0x7f73d40585e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f73db755640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d4163930 con 0x7f73d40b9f80 2026-02-25T15:01:42.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f73db755640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d4163bb0 con 0x7f73d40ba350 2026-02-25T15:01:42.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f73db755640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d4163e90 con 0x7f73d40585e0 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f73db755640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f73db755640 10 monclient(hunting): authenticate will time out at 2991.713099s 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a8f7fe640 1 -- 10.20.193.90:0/3406671501 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1481593848 0 0) 0x7f2a78003610 con 0x7f2a980b9f80 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a8f7fe640 1 -- 10.20.193.90:0/3406671501 >> v1:10.20.193.93:6789/0 conn(0x7f2a980ba350 legacy=0x7f2a98057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a8f7fe640 1 -- 10.20.193.90:0/3406671501 >> v1:10.20.193.90:6789/0 conn(0x7f2a980585e0 legacy=0x7f2a98166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a8f7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a8f7fe640 1 monclient: found mon.noname-c 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a8f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a8f7fe640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a8f7fe640 1 -- 10.20.193.90:0/3406671501 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a980573f0 con 0x7f2a980b9f80 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a8f7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a8f7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a8f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a8f7fe640 1 -- 10.20.193.90:0/3406671501 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2a88002df0 con 0x7f2a980b9f80 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a8f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a8f7fe640 10 monclient: dump: 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.159 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a9ec55640 5 monclient: authenticate success, global_id 4397 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f2a9ec55640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7fc84ff2e640 1 -- start start 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7fc84ff2e640 10 monclient: init 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7fc84ff2e640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7fc84ff2e640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7fc84ff2e640 10 monclient(hunting): picked mon.noname-a con 0x7fc84815b0a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7fc84ff2e640 10 monclient(hunting): picked mon.noname-c con 0x7fc8480b3190 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7fc84ff2e640 10 monclient(hunting): picked mon.noname-b con 0x7fc8480b5da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7fc84ff2e640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8480b3610 con 0x7fc84815b0a0 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7fc84ff2e640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8480b38f0 con 0x7fc8480b5da0 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7fc84ff2e640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8480b3bd0 con 0x7fc8480b3190 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7fc84ff2e640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7fc84ff2e640 10 monclient(hunting): authenticate will time out at 2991.713796s 2026-02-25T15:01:42.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f0bfe30e640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f0bf815d0c0 0x7f0bf80b0590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44762/0 (socket says 10.20.193.90:44762) 2026-02-25T15:01:42.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.113+0000 7f0bfe30e640 1 -- 10.20.193.90:0/3350704169 learned_addr learned my addr 10.20.193.90:0/3350704169 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.114+0000 7f26d49ce640 1 -- 10.20.193.90:0/838923256 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f26c4003000 con 0x7f26d0157b50 2026-02-25T15:01:42.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.114+0000 7f26d49ce640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.114+0000 7f26d49ce640 1 -- 10.20.193.90:0/838923256 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f26c4003400 con 0x7f26d0157b50 2026-02-25T15:01:42.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.114+0000 7f26d7c5b640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.114+0000 7f26d7c5b640 4 set_mon_vals no callback set 2026-02-25T15:01:42.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.114+0000 7f26d49ce640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.114+0000 7f26d49ce640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.114+0000 7f26d49ce640 10 monclient: dump: 2026-02-25T15:01:42.161 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.161 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.161 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.161 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.161 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.162 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.162 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.162 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.162 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.162 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.162 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.115+0000 7f4f7bfff640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f4f7c15b220 0x7f4f7c058d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46366/0 (socket says 10.20.193.90:46366) 2026-02-25T15:01:42.162 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.115+0000 7f4f7bfff640 1 -- 10.20.193.90:0/427355895 learned_addr learned my addr 10.20.193.90:0/427355895 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.162 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.115+0000 7f26d7c5b640 10 monclient: shutdown 2026-02-25T15:01:42.162 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.115+0000 7f26d7c5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.162 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.115+0000 7f26d7c5b640 1 -- 10.20.193.90:0/838923256 >> v1:10.20.193.94:6789/0 conn(0x7f26d0157b50 legacy=0x7f26d01645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.162 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.115+0000 7f5b96c15640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f5b900585e0 0x7f5b90166cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44798/0 (socket says 10.20.193.90:44798) 2026-02-25T15:01:42.162 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.115+0000 7f5b96c15640 1 -- 10.20.193.90:0/1899139546 learned_addr learned my addr 10.20.193.90:0/1899139546 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.115+0000 7f26d7c5b640 1 -- 10.20.193.90:0/838923256 shutdown_connections 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.115+0000 7f26d7c5b640 1 -- 10.20.193.90:0/838923256 >> 10.20.193.90:0/838923256 conn(0x7f26d00b2ef0 msgr2=0x7f26d00b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.115+0000 7f26d7c5b640 1 -- 10.20.193.90:0/838923256 shutdown_connections 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.115+0000 7f26d7c5b640 1 -- 10.20.193.90:0/838923256 wait complete. 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f26d7c5b640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f26d7c5b640 10 monclient: monmap: 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f26d7c5b640 1 librados: starting msgr at 2026-02-25T15:01:42.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f26d7c5b640 1 librados: starting objecter 2026-02-25T15:01:42.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f26d7c5b640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f26d7c5b640 1 Processor -- start 2026-02-25T15:01:42.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f26d7c5b640 1 -- start start 2026-02-25T15:01:42.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f26d7c5b640 1 librados: setting wanted keys 2026-02-25T15:01:42.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f26d7c5b640 1 librados: calling monclient init 2026-02-25T15:01:42.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f26d7c5b640 10 monclient: init 2026-02-25T15:01:42.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f5147fff640 1 -- 10.20.193.90:0/2272946097 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3644774712 0 0) 0x7f5158163e90 con 0x7f51580b9f80 2026-02-25T15:01:42.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f5147fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f5147fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f5147fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f5147fff640 10 monclient(hunting): my global_id is 4406 2026-02-25T15:01:42.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.116+0000 7f5147fff640 1 -- 10.20.193.90:0/2272946097 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f514c002910 con 0x7f51580b9f80 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f26d7c5b640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f26d7c5b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f26d7c5b640 10 monclient(hunting): picked mon.noname-a con 0x7f26d0157780 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f26d7c5b640 10 monclient(hunting): picked mon.noname-c con 0x7f26d0157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f26d7c5b640 10 monclient(hunting): picked mon.noname-b con 0x7f26d0164d00 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f26d7c5b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d012a110 con 0x7f26d0157780 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f26d7c5b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d012a3f0 con 0x7f26d0164d00 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f26d7c5b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d012a6d0 con 0x7f26d0157b50 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f26d7c5b640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f26d7c5b640 10 monclient(hunting): authenticate will time out at 2991.717502s 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f47c9ffb640 1 -- 10.20.193.90:0/1900780581 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2172922182 0 0) 0x7f47cc163bb0 con 0x7f47cc0585e0 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f47c9ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f47c9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f47c9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f47c9ffb640 10 monclient(hunting): my global_id is 4414 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f47c9ffb640 1 -- 10.20.193.90:0/1900780581 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47cc163e90 con 0x7f47cc0585e0 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f5126ffd640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f51280b9f80 0x7f5128163220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46378/0 (socket says 10.20.193.90:46378) 2026-02-25T15:01:42.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7f5126ffd640 1 -- 10.20.193.90:0/1923812045 learned_addr learned my addr 10.20.193.90:0/1923812045 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7faaedffb640 1 -- 10.20.193.90:0/2496819441 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7faae4003000 con 0x7faaf0155b20 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7faaedffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7faaedffb640 1 -- 10.20.193.90:0/2496819441 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7faae4003440 con 0x7faaf0155b20 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7faaedffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7faaedffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7faaedffb640 10 monclient: dump: 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7faaf52d1640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.117+0000 7faaf52d1640 4 set_mon_vals no callback set 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f5147fff640 1 -- 10.20.193.90:0/2272946097 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3594410116 0 0) 0x7f5140003610 con 0x7f51580585e0 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f5147fff640 1 -- 10.20.193.90:0/2272946097 >> v1:10.20.193.94:6789/0 conn(0x7f51580b9f80 legacy=0x7f5158163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f5147fff640 1 -- 10.20.193.90:0/2272946097 >> v1:10.20.193.90:6789/0 conn(0x7f51580ba350 legacy=0x7f5158057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f5147fff640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f5147fff640 1 monclient: found mon.noname-b 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f5147fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f5147fff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f5147fff640 1 -- 10.20.193.90:0/2272946097 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51580573f0 con 0x7f51580585e0 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f5147fff640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f7b9cda2640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f7b9cda2640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f7b9cda2640 10 monclient: monmap: 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.118959+0000 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.118959+0000 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.167 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7faaf52d1640 10 monclient: shutdown 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7faaf52d1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7faaf52d1640 1 -- 10.20.193.90:0/2496819441 >> v1:10.20.193.93:6789/0 conn(0x7faaf0155b20 legacy=0x7faaf00b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f5147fff640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f5147fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f5147fff640 1 -- 10.20.193.90:0/2272946097 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5154003170 con 0x7f51580585e0 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f5147fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f5147fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f5147fff640 10 monclient: dump: 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f515f98a640 5 monclient: authenticate success, global_id 4408 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f515f98a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7faaf52d1640 1 -- 10.20.193.90:0/2496819441 shutdown_connections 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7faaf52d1640 1 -- 10.20.193.90:0/2496819441 >> 10.20.193.90:0/2496819441 conn(0x7faaf00b2ef0 msgr2=0x7faaf00b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7faaf52d1640 1 -- 10.20.193.90:0/2496819441 shutdown_connections 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7faaf52d1640 1 -- 10.20.193.90:0/2496819441 wait complete. 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f7b9cda2640 1 Processor -- start 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f1b808e9640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f1b808e9640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.118+0000 7f1b808e9640 10 monclient: monmap: 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.119529+0000 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.119529+0000 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.169 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7f7b9cda2640 1 -- start start 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7f7b9cda2640 10 monclient: init 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7f1b808e9640 1 Processor -- start 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7f1b808e9640 1 -- start start 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7f1b808e9640 10 monclient: init 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7f7b9cda2640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7f7b9cda2640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7f7b9cda2640 10 monclient(hunting): picked mon.noname-c con 0x7f7b980b9f80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7f7b9cda2640 10 monclient(hunting): picked mon.noname-b con 0x7f7b980ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7f7b9cda2640 10 monclient(hunting): picked mon.noname-a con 0x7f7b980585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7f7b9cda2640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b98163930 con 0x7f7b980585e0 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7f7b9cda2640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b98163bb0 con 0x7f7b980ba350 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7f7b9cda2640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b98163e90 con 0x7f7b980b9f80 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7f7b9cda2640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7f7b9cda2640 10 monclient(hunting): authenticate will time out at 2991.719623s 2026-02-25T15:01:42.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7faaf52d1640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.119+0000 7faaf52d1640 10 monclient: monmap: 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 1 librados: starting msgr at 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 1 librados: starting objecter 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f1b808e9640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f1b808e9640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f82aecd0640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f82aecd0640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 1 Processor -- start 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f1b808e9640 10 monclient(hunting): picked mon.noname-a con 0x7f1b780b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f82aecd0640 10 monclient: monmap: 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.120711+0000 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.120711+0000 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 1 -- start start 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 1 librados: setting wanted keys 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 1 librados: calling monclient init 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 10 monclient: init 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f1b808e9640 10 monclient(hunting): picked mon.noname-b con 0x7f1b780ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:42.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 10 monclient(hunting): picked mon.noname-b con 0x7faaf0056c70 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 10 monclient(hunting): picked mon.noname-a con 0x7faaf0155b20 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 10 monclient(hunting): picked mon.noname-c con 0x7faaf0162f10 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf0129f30 con 0x7faaf0155b20 2026-02-25T15:01:42.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf012a210 con 0x7faaf0056c70 2026-02-25T15:01:42.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf012a4f0 con 0x7faaf0162f10 2026-02-25T15:01:42.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7faaf52d1640 10 monclient(hunting): authenticate will time out at 2991.720560s 2026-02-25T15:01:42.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f1b808e9640 10 monclient(hunting): picked mon.noname-c con 0x7f1b780585e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f1b808e9640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b78163930 con 0x7f1b780b9f80 2026-02-25T15:01:42.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f1b808e9640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b78163bb0 con 0x7f1b780ba350 2026-02-25T15:01:42.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f1b808e9640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b78163e90 con 0x7f1b780585e0 2026-02-25T15:01:42.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f1b808e9640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f1b808e9640 10 monclient(hunting): authenticate will time out at 2991.720531s 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f82aecd0640 1 Processor -- start 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f82aecd0640 1 -- start start 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.120+0000 7f82aecd0640 10 monclient: init 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.121+0000 7f82aecd0640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.121+0000 7f82aecd0640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.121+0000 7f82aecd0640 10 monclient(hunting): picked mon.noname-b con 0x7f82a8159d10 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.121+0000 7f82aecd0640 10 monclient(hunting): picked mon.noname-a con 0x7f82a805d990 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.121+0000 7f82aecd0640 10 monclient(hunting): picked mon.noname-c con 0x7f82a805a310 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.121+0000 7f82aecd0640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82a81635c0 con 0x7f82a805d990 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.121+0000 7f82aecd0640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82a81638a0 con 0x7f82a8159d10 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.121+0000 7f82aecd0640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82a8163b80 con 0x7f82a805a310 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.121+0000 7f82aecd0640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.121+0000 7f82aecd0640 10 monclient(hunting): authenticate will time out at 2991.721293s 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee07fff640 1 -- 10.20.193.90:0/3928160845 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3567963222 0 0) 0x7fee18160b00 con 0x7fee18058580 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee07fff640 1 -- 10.20.193.90:0/3928160845 >> v1:10.20.193.94:6789/0 conn(0x7fee18059e70 legacy=0x7fee1815fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee07fff640 1 -- 10.20.193.90:0/3928160845 >> v1:10.20.193.90:6789/0 conn(0x7fee1805a240 legacy=0x7fee18156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee07fff640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee07fff640 1 monclient: found mon.noname-b 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee07fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee07fff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee07fff640 1 -- 10.20.193.90:0/3928160845 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee18055b20 con 0x7fee18058580 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee07fff640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee07fff640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee07fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee07fff640 1 -- 10.20.193.90:0/3928160845 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fee0c002df0 con 0x7fee18058580 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee07fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee07fff640 10 monclient: dump: 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee1f4c9640 5 monclient: authenticate success, global_id 4411 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7fee1f4c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7f4f7affd640 1 -- 10.20.193.90:0/427355895 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2492662894 0 0) 0x7f4f7c1653f0 con 0x7f4f7c15b220 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7f4f7affd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7f4f7affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7f4f7affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7f4f7affd640 10 monclient(hunting): my global_id is 4417 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.122+0000 7f4f7affd640 1 -- 10.20.193.90:0/427355895 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f60003610 con 0x7f4f7c15b220 2026-02-25T15:01:42.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f5147fff640 1 -- 10.20.193.90:0/2272946097 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5154003340 con 0x7f51580585e0 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f5147fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f5147fff640 1 -- 10.20.193.90:0/2272946097 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5154003780 con 0x7f51580585e0 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f5147fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f5147fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f5147fff640 10 monclient: dump: 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f515f98a640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f515f98a640 4 set_mon_vals no callback set 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f0bfd30c640 1 -- 10.20.193.90:0/3350704169 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1807328238 0 0) 0x7f0bf80b5da0 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f0bfd30c640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f0bfd30c640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f0bfd30c640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f0bfd30c640 10 monclient(hunting): my global_id is 4420 2026-02-25T15:01:42.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f0bfd30c640 1 -- 10.20.193.90:0/3350704169 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0be0003610 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f515f98a640 10 monclient: shutdown 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f515f98a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f515f98a640 1 -- 10.20.193.90:0/2272946097 >> v1:10.20.193.93:6789/0 conn(0x7f51580585e0 legacy=0x7f5158166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f515f98a640 1 -- 10.20.193.90:0/2272946097 shutdown_connections 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f515f98a640 1 -- 10.20.193.90:0/2272946097 >> 10.20.193.90:0/2272946097 conn(0x7f51580b7990 msgr2=0x7f51580b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f515f98a640 1 -- 10.20.193.90:0/2272946097 shutdown_connections 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f515f98a640 1 -- 10.20.193.90:0/2272946097 wait complete. 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.123+0000 7f5b95412640 1 -- 10.20.193.90:0/1899139546 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2492074101 0 0) 0x7f5b90163bb0 con 0x7f5b900b9f80 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f5b95412640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f5b95412640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f5b95412640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f5b95412640 10 monclient(hunting): my global_id is 4426 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f5b95412640 1 -- 10.20.193.90:0/1899139546 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b78003610 con 0x7f5b900b9f80 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f26d61d1640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f26d0164d00 0x7f26d00a9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46418/0 (socket says 10.20.193.90:46418) 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f26d61d1640 1 -- 10.20.193.90:0/3380070399 learned_addr learned my addr 10.20.193.90:0/3380070399 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47c9ffb640 1 -- 10.20.193.90:0/1900780581 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 874395146 0 0) 0x7f47b8003610 con 0x7f47cc0b9f80 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47c9ffb640 1 -- 10.20.193.90:0/1900780581 >> v1:10.20.193.93:6789/0 conn(0x7f47cc0585e0 legacy=0x7f47cc166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47c9ffb640 1 -- 10.20.193.90:0/1900780581 >> v1:10.20.193.90:6789/0 conn(0x7f47cc0ba350 legacy=0x7f47cc057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47c9ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47c9ffb640 1 monclient: found mon.noname-c 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47c9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47c9ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47c9ffb640 1 -- 10.20.193.90:0/1900780581 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc0573f0 con 0x7f47cc0b9f80 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47c9ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47c9ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47c9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47c9ffb640 1 -- 10.20.193.90:0/1900780581 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f47c0003170 con 0x7f47cc0b9f80 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47c9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47c9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47c9ffb640 10 monclient: dump: 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7fc0456e6640 1 -- 10.20.193.90:0/3278426197 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc02c002fc0 con 0x7fc040161a10 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7fc0456e6640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7fc0456e6640 1 -- 10.20.193.90:0/3278426197 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fc02c003400 con 0x7fc040161a10 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7fc0456e6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7fc0456e6640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7fc0456e6640 10 monclient: dump: 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.178 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7fc048973640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7fc048973640 4 set_mon_vals no callback set 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 10 monclient: monmap: 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47d1513640 5 monclient: authenticate success, global_id 4403 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f47d1513640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 1 librados: starting msgr at 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 1 librados: starting objecter 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 1 Processor -- start 2026-02-25T15:01:42.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f73d94ca640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f73d40b9f80 0x7f73d4163220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44804/0 (socket says 10.20.193.90:44804) 2026-02-25T15:01:42.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f73d94ca640 1 -- 10.20.193.90:0/81938051 learned_addr learned my addr 10.20.193.90:0/81938051 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 1 -- start start 2026-02-25T15:01:42.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 1 librados: setting wanted keys 2026-02-25T15:01:42.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 1 librados: calling monclient init 2026-02-25T15:01:42.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 10 monclient: init 2026-02-25T15:01:42.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f26be7fc640 10 client.?.objecter ms_handle_connect 0x7f26d0164d00 2026-02-25T15:01:42.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f26be7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7fc048973640 10 monclient: shutdown 2026-02-25T15:01:42.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7fc048973640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7fc048973640 1 -- 10.20.193.90:0/3278426197 >> v1:10.20.193.94:6789/0 conn(0x7fc040161a10 legacy=0x7fc0400b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:42.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7fc048973640 1 -- 10.20.193.90:0/3278426197 shutdown_connections 2026-02-25T15:01:42.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7fc048973640 1 -- 10.20.193.90:0/3278426197 >> 10.20.193.90:0/3278426197 conn(0x7fc0400b5290 msgr2=0x7fc0400b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 10 monclient(hunting): picked mon.noname-c con 0x7f51580585e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7fc048973640 1 -- 10.20.193.90:0/3278426197 shutdown_connections 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7fc048973640 1 -- 10.20.193.90:0/3278426197 wait complete. 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 10 monclient(hunting): picked mon.noname-a con 0x7f51580b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 10 monclient(hunting): picked mon.noname-b con 0x7f51580ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5158159ea0 con 0x7f51580b9f80 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f4f7affd640 1 -- 10.20.193.90:0/427355895 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3055436023 0 0) 0x7f4f7c165110 con 0x7f4f7c055f60 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f4f7affd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f4f7affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f4f7affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f4f7affd640 10 monclient(hunting): my global_id is 4404 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f4f7affd640 1 -- 10.20.193.90:0/427355895 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f7c1653f0 con 0x7f4f7c055f60 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.124+0000 7f515f98a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51581fb450 con 0x7f51580ba350 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f515f98a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51581fb730 con 0x7f51580585e0 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f515f98a640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f515f98a640 10 monclient(hunting): authenticate will time out at 2991.725027s 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f4f7affd640 1 -- 10.20.193.90:0/427355895 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3689138978 0 0) 0x7f4f7c161050 con 0x7f4f7c059470 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f4f7affd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f4f7affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f2a8f7fe640 1 -- 10.20.193.90:0/3406671501 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2a88003000 con 0x7f2a980b9f80 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f2a8f7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f2a8f7fe640 1 -- 10.20.193.90:0/3406671501 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2a88003440 con 0x7f2a980b9f80 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f2a8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f2a8f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f2a8f7fe640 10 monclient: dump: 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f2a9ec55640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f2a9ec55640 4 set_mon_vals no callback set 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f4f7affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f4f7affd640 10 monclient(hunting): my global_id is 4418 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f4f7affd640 1 -- 10.20.193.90:0/427355895 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f7c165110 con 0x7f4f7c059470 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7fc048973640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7fc048973640 10 monclient: monmap: 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.182 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f5125ffb640 1 -- 10.20.193.90:0/1923812045 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1969160106 0 0) 0x7f5128163e90 con 0x7f51280585e0 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f5125ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f5125ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f5125ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f5125ffb640 10 monclient(hunting): my global_id is 4415 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f5125ffb640 1 -- 10.20.193.90:0/1923812045 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5114003610 con 0x7f51280585e0 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7fc048973640 1 librados: starting msgr at 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7fc048973640 1 librados: starting objecter 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f2a9ec55640 10 monclient: shutdown 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f2a9ec55640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7f2a9ec55640 1 -- 10.20.193.90:0/3406671501 >> v1:10.20.193.94:6789/0 conn(0x7f2a980b9f80 legacy=0x7f2a98163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7fc048973640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.125+0000 7fc048973640 1 Processor -- start 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7fc048973640 1 -- start start 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7fc048973640 1 librados: setting wanted keys 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7fc048973640 1 librados: calling monclient init 2026-02-25T15:01:42.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7fc048973640 10 monclient: init 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7f2a9ec55640 1 -- 10.20.193.90:0/3406671501 shutdown_connections 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7f2a9ec55640 1 -- 10.20.193.90:0/3406671501 >> 10.20.193.90:0/3406671501 conn(0x7f2a980b7990 msgr2=0x7f2a980b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7f2a9ec55640 1 -- 10.20.193.90:0/3406671501 shutdown_connections 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7f2a9ec55640 1 -- 10.20.193.90:0/3406671501 wait complete. 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7fc048973640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7fc048973640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7fc048973640 10 monclient(hunting): picked mon.noname-c con 0x7fc0400b5da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7fc048973640 10 monclient(hunting): picked mon.noname-b con 0x7fc040161640 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7fc048973640 10 monclient(hunting): picked mon.noname-a con 0x7fc040161a10 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7fc048973640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc04013a4b0 con 0x7fc040161a10 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7fc048973640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc04013a790 con 0x7fc040161640 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7fc048973640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc04013aa70 con 0x7fc0400b5da0 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7fc048973640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7fc048973640 10 monclient(hunting): authenticate will time out at 2991.726515s 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7f2a9ec55640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7f2a9ec55640 10 monclient: monmap: 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.184 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7f26be7fc640 10 client.?.objecter ms_handle_connect 0x7f26d0157780 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7f26be7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7f1b7ee5f640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f1b780585e0 0x7f1b78166cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54134/0 (socket says 10.20.193.90:54134) 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7f1b7ee5f640 1 -- 10.20.193.90:0/1408243345 learned_addr learned my addr 10.20.193.90:0/1408243345 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7f2a9ec55640 1 librados: starting msgr at 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7f2a9ec55640 1 librados: starting objecter 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7f2a9ec55640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.126+0000 7f2a9ec55640 1 Processor -- start 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f2a9ec55640 1 -- start start 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f2a9ec55640 1 librados: setting wanted keys 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f2a9ec55640 1 librados: calling monclient init 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f2a9ec55640 10 monclient: init 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f7b95d74640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f7b980ba350 0x7f7b98057ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46422/0 (socket says 10.20.193.90:46422) 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f7b95d74640 1 -- 10.20.193.90:0/3051108113 learned_addr learned my addr 10.20.193.90:0/3051108113 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7faaee7fc640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7faaf0155b20 0x7faaf005ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44842/0 (socket says 10.20.193.90:44842) 2026-02-25T15:01:42.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7faaee7fc640 1 -- 10.20.193.90:0/2340633913 learned_addr learned my addr 10.20.193.90:0/2340633913 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f0bfd30c640 1 -- 10.20.193.90:0/3350704169 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3683490633 0 0) 0x7f0bf80b6080 con 0x7f0bf80b28b0 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f0bfd30c640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f0bfd30c640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f0bfd30c640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f0bfd30c640 10 monclient(hunting): my global_id is 4409 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f0bfd30c640 1 -- 10.20.193.90:0/3350704169 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bf80b5da0 con 0x7f0bf80b28b0 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f2a9ec55640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f2a9ec55640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f2a9ec55640 10 monclient(hunting): picked mon.noname-a con 0x7f2a980585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f2a9ec55640 10 monclient(hunting): picked mon.noname-c con 0x7f2a980b9f80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f2a9ec55640 10 monclient(hunting): picked mon.noname-b con 0x7f2a980ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f2a9ec55640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a98156bc0 con 0x7f2a980585e0 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f2a9ec55640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a981ff450 con 0x7f2a980ba350 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f2a9ec55640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a981fb140 con 0x7f2a980b9f80 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f2a9ec55640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f2a9ec55640 10 monclient(hunting): authenticate will time out at 2991.727745s 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f0bfd30c640 1 -- 10.20.193.90:0/3350704169 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3568366675 0 0) 0x7f0bf80b8540 con 0x7f0bf815d0c0 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f0bfd30c640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f0bfd30c640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f0bfd30c640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f0bfd30c640 10 monclient(hunting): my global_id is 4398 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.127+0000 7f0bfd30c640 1 -- 10.20.193.90:0/3350704169 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bf80b6080 con 0x7f0bf815d0c0 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7fc84d4a2640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fc8480b3190 0x7fc8480b7720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54102/0 (socket says 10.20.193.90:54102) 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7fc84d4a2640 1 -- 10.20.193.90:0/110573329 learned_addr learned my addr 10.20.193.90:0/110573329 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5125ffb640 1 -- 10.20.193.90:0/1923812045 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1007815855 0 0) 0x7f5128163bb0 con 0x7f51280b9f80 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5125ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5125ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5125ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5125ffb640 10 monclient(hunting): my global_id is 4423 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5125ffb640 1 -- 10.20.193.90:0/1923812045 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5128163e90 con 0x7f51280b9f80 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7faacf7fe640 10 client.?.objecter ms_handle_connect 0x7faaf0155b20 2026-02-25T15:01:42.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7faacf7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5b95412640 1 -- 10.20.193.90:0/1899139546 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1165673715 0 0) 0x7f5b90163e90 con 0x7f5b900ba350 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5b95412640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5b95412640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5b95412640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5b95412640 10 monclient(hunting): my global_id is 4412 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5b95412640 1 -- 10.20.193.90:0/1899139546 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b90163bb0 con 0x7f5b900ba350 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5b95412640 1 -- 10.20.193.90:0/1899139546 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3978014803 0 0) 0x7f5b8c002c90 con 0x7f5b900585e0 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5b95412640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5b95412640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5b95412640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5b95412640 10 monclient(hunting): my global_id is 4401 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.128+0000 7f5b95412640 1 -- 10.20.193.90:0/1899139546 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b90163e90 con 0x7f5b900585e0 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.129+0000 7f515d6ff640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f51580585e0 0x7f5158159790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54156/0 (socket says 10.20.193.90:54156) 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.129+0000 7f515d6ff640 1 -- 10.20.193.90:0/2774738662 learned_addr learned my addr 10.20.193.90:0/2774738662 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.130+0000 7f5145ffb640 10 client.?.objecter ms_handle_connect 0x7f51580b9f80 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.130+0000 7f5145ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.130+0000 7faacf7fe640 10 client.?.objecter ms_handle_connect 0x7faaf0162f10 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.130+0000 7faacf7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.130+0000 7f5145ffb640 10 client.?.objecter ms_handle_connect 0x7f51580585e0 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.130+0000 7f5145ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.131+0000 7f26be7fc640 10 client.?.objecter ms_handle_connect 0x7f26d0157b50 2026-02-25T15:01:42.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.131+0000 7f26be7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7fee07fff640 1 -- 10.20.193.90:0/3928160845 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fee0c002fc0 con 0x7fee18058580 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7fee07fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7fee07fff640 1 -- 10.20.193.90:0/3928160845 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fee0c003400 con 0x7fee18058580 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7fee07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7fee07fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7fee07fff640 10 monclient: dump: 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7fee1f4c9640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7fee1f4c9640 4 set_mon_vals no callback set 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7fee1f4c9640 10 monclient: shutdown 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7fee1f4c9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7fee1f4c9640 1 -- 10.20.193.90:0/3928160845 >> v1:10.20.193.93:6789/0 conn(0x7fee18058580 legacy=0x7fee18056600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7fee1f4c9640 1 -- 10.20.193.90:0/3928160845 shutdown_connections 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7fee1f4c9640 1 -- 10.20.193.90:0/3928160845 >> 10.20.193.90:0/3928160845 conn(0x7fee180590f0 msgr2=0x7fee180594f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7fee1f4c9640 1 -- 10.20.193.90:0/3928160845 shutdown_connections 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7f82aca45640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f82a8159d10 0x7f82a805d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46458/0 (socket says 10.20.193.90:46458) 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7f82aca45640 1 -- 10.20.193.90:0/241044117 learned_addr learned my addr 10.20.193.90:0/241044117 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.132+0000 7fee1f4c9640 1 -- 10.20.193.90:0/3928160845 wait complete. 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f47c9ffb640 1 -- 10.20.193.90:0/1900780581 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f47c0003380 con 0x7f47cc0b9f80 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f47c9ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f47c9ffb640 1 -- 10.20.193.90:0/1900780581 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f47c00037c0 con 0x7f47cc0b9f80 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f47c9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f47c9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f47c9ffb640 10 monclient: dump: 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f47d1513640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f47d1513640 4 set_mon_vals no callback set 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7fee1f4c9640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7fee1f4c9640 10 monclient: monmap: 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7fee1f4c9640 1 librados: starting msgr at 2026-02-25T15:01:42.190 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7fee1f4c9640 1 librados: starting objecter 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f1b7d65c640 1 -- 10.20.193.90:0/1408243345 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3382933213 0 0) 0x7f1b70002910 con 0x7f1b780b9f80 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f1b7d65c640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f1b7d65c640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f1b7d65c640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f1b7d65c640 10 monclient(hunting): my global_id is 4413 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f1b7d65c640 1 -- 10.20.193.90:0/1408243345 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b68003610 con 0x7f1b780b9f80 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7fee1f4c9640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7fee1f4c9640 1 Processor -- start 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7fee1f4c9640 1 -- start start 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7fee1f4c9640 1 librados: setting wanted keys 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7fee1f4c9640 1 librados: calling monclient init 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7fee1f4c9640 10 monclient: init 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f47d1513640 10 monclient: shutdown 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f47d1513640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f47d1513640 1 -- 10.20.193.90:0/1900780581 >> v1:10.20.193.94:6789/0 conn(0x7f47cc0b9f80 legacy=0x7f47cc163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f47d1513640 1 -- 10.20.193.90:0/1900780581 shutdown_connections 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f47d1513640 1 -- 10.20.193.90:0/1900780581 >> 10.20.193.90:0/1900780581 conn(0x7f47cc0b7990 msgr2=0x7f47cc0b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.133+0000 7f47d1513640 1 -- 10.20.193.90:0/1900780581 shutdown_connections 2026-02-25T15:01:42.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f47d1513640 1 -- 10.20.193.90:0/1900780581 wait complete. 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7fee1f4c9640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7fee1f4c9640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7fee1f4c9640 10 monclient(hunting): picked mon.noname-b con 0x7fee18058580 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7fee1f4c9640 10 monclient(hunting): picked mon.noname-c con 0x7fee18059e70 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7fee1f4c9640 10 monclient(hunting): picked mon.noname-a con 0x7fee1805a240 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7fee1f4c9640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee18134c40 con 0x7fee1805a240 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7fee1f4c9640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee18134f20 con 0x7fee18058580 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7fee1f4c9640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee18135200 con 0x7fee18059e70 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7fee1f4c9640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7fee1f4c9640 10 monclient(hunting): authenticate will time out at 2991.734315s 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f47d1513640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f47d1513640 10 monclient: monmap: 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f47d1513640 1 librados: starting msgr at 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f47d1513640 1 librados: starting objecter 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f47d1513640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f47d1513640 1 Processor -- start 2026-02-25T15:01:42.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f4f7affd640 1 -- 10.20.193.90:0/427355895 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2879973576 0 0) 0x7f4f60003610 con 0x7f4f7c15b220 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f4f7affd640 1 -- 10.20.193.90:0/427355895 >> v1:10.20.193.94:6789/0 conn(0x7f4f7c059470 legacy=0x7f4f7c1a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f4f7affd640 1 -- 10.20.193.90:0/427355895 >> v1:10.20.193.90:6789/0 conn(0x7f4f7c055f60 legacy=0x7f4f7c1c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3939615547 0 0) 0x7f26d012a3f0 con 0x7f26d0164d00 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f26be7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f26be7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f26be7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f26be7fc640 10 monclient(hunting): my global_id is 4429 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f4f7affd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f4f7affd640 1 monclient: found mon.noname-b 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f4f7affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f4f7affd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f4f7affd640 1 -- 10.20.193.90:0/427355895 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7c161330 con 0x7f4f7c15b220 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f47d1513640 1 -- start start 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f47d1513640 1 librados: setting wanted keys 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f47d1513640 1 librados: calling monclient init 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.134+0000 7f47d1513640 10 monclient: init 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26a0003610 con 0x7f26d0164d00 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f4f7affd640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f4f7affd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f4f7affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f4f7affd640 1 -- 10.20.193.90:0/427355895 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4f68002df0 con 0x7f4f7c15b220 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f4f7affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f4f7affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f4f7affd640 10 monclient: dump: 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f4f826b1640 5 monclient: authenticate success, global_id 4417 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f4f826b1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1033022419 0 0) 0x7f5158159ea0 con 0x7f51580b9f80 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f5145ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f5145ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f5145ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f5145ffb640 10 monclient(hunting): my global_id is 4419 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f47d1513640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f47d1513640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5128003610 con 0x7f51580b9f80 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f47d1513640 10 monclient(hunting): picked mon.noname-b con 0x7f47cc0585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f47d1513640 10 monclient(hunting): picked mon.noname-a con 0x7f47cc0b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f47d1513640 10 monclient(hunting): picked mon.noname-c con 0x7f47cc0ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f47d1513640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47cc1fbcb0 con 0x7f47cc0b9f80 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f47d1513640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47cc162880 con 0x7f47cc0585e0 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f47d1513640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47cc162b60 con 0x7f47cc0ba350 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f47d1513640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f47d1513640 10 monclient(hunting): authenticate will time out at 2991.735464s 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f2a9c9ca640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f2a980585e0 0x7f2a980b4260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44872/0 (socket says 10.20.193.90:44872) 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.135+0000 7f2a9c9ca640 1 -- 10.20.193.90:0/1700082558 learned_addr learned my addr 10.20.193.90:0/1700082558 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0bfd30c640 1 -- 10.20.193.90:0/3350704169 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1682688125 0 0) 0x7f0be0003610 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0bfd30c640 1 -- 10.20.193.90:0/3350704169 >> v1:10.20.193.94:6789/0 conn(0x7f0bf80b28b0 legacy=0x7f0bf80b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0bfd30c640 1 -- 10.20.193.90:0/3350704169 >> v1:10.20.193.90:6789/0 conn(0x7f0bf815d0c0 legacy=0x7f0bf80b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0bfd30c640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0bfd30c640 1 monclient: found mon.noname-b 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0bfd30c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0bfd30c640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0bfd30c640 1 -- 10.20.193.90:0/3350704169 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf8163e10 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0bfd30c640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0bfd30c640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0bfd30c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0bfd30c640 1 -- 10.20.193.90:0/3350704169 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0bec002df0 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0c00599640 5 monclient: authenticate success, global_id 4420 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0c00599640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0bfd30c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0bfd30c640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0bfd30c640 10 monclient: dump: 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.195 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.196 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.196 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f0c00599640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f1b7d65c640 1 -- 10.20.193.90:0/1408243345 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 859246412 0 0) 0x7f1b78163e90 con 0x7f1b780585e0 2026-02-25T15:01:42.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f1b7d65c640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f1b7d65c640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f1b7d65c640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f1b7d65c640 10 monclient(hunting): my global_id is 4424 2026-02-25T15:01:42.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f1b7d65c640 1 -- 10.20.193.90:0/1408243345 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b70002910 con 0x7f1b780585e0 2026-02-25T15:01:42.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f5b95412640 1 -- 10.20.193.90:0/1899139546 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3862538378 0 0) 0x7f5b78003610 con 0x7f5b900b9f80 2026-02-25T15:01:42.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f5b95412640 1 -- 10.20.193.90:0/1899139546 >> v1:10.20.193.94:6789/0 conn(0x7f5b900ba350 legacy=0x7f5b90057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f5b95412640 1 -- 10.20.193.90:0/1899139546 >> v1:10.20.193.90:6789/0 conn(0x7f5b900585e0 legacy=0x7f5b90166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f5b95412640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f5b95412640 1 monclient: found mon.noname-b 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f5b95412640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f5b95412640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f5b95412640 1 -- 10.20.193.90:0/1899139546 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b900573f0 con 0x7f5b900b9f80 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f5b95412640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f5b95412640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f5b95412640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f5b95412640 1 -- 10.20.193.90:0/1899139546 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5b80002df0 con 0x7f5b900b9f80 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f5b9869f640 5 monclient: authenticate success, global_id 4426 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.136+0000 7f5b9869f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f5b95412640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f5b95412640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f5b95412640 10 monclient: dump: 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f5b9869f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f73c3fff640 1 -- 10.20.193.90:0/81938051 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 209625194 0 0) 0x7f73c4002910 con 0x7f73d40b9f80 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f73c3fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f73c3fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f73c3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f73c3fff640 10 monclient(hunting): my global_id is 4407 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f73c3fff640 1 -- 10.20.193.90:0/81938051 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73bc003610 con 0x7f73d40b9f80 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f2a8d7fa640 10 client.?.objecter ms_handle_connect 0x7f2a980585e0 2026-02-25T15:01:42.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f2a8d7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3800042400 0 0) 0x7f26d012a110 con 0x7f26d0157780 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f26be7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f26be7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f26be7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f26be7fc640 10 monclient(hunting): my global_id is 4410 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26d012a3f0 con 0x7f26d0157780 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4124656264 0 0) 0x7faaf0129f30 con 0x7faaf0155b20 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7faacf7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7faacf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7faacf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7faacf7fe640 10 monclient(hunting): my global_id is 4422 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.137+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faabc003610 con 0x7faaf0155b20 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.138+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2107385516 0 0) 0x7f51581fb730 con 0x7f51580585e0 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.138+0000 7f5145ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.138+0000 7f5145ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.138+0000 7f5145ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.138+0000 7f5145ffb640 10 monclient(hunting): my global_id is 4433 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.138+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5158159ea0 con 0x7f51580585e0 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.139+0000 7f7b95573640 1 -- 10.20.193.90:0/3051108113 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3356177067 0 0) 0x7f7b98163bb0 con 0x7f7b980ba350 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.139+0000 7f7b95573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.139+0000 7f7b95573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.139+0000 7f7b95573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.139+0000 7f7b95573640 10 monclient(hunting): my global_id is 4432 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.139+0000 7faacf7fe640 10 client.?.objecter ms_handle_connect 0x7faaf0056c70 2026-02-25T15:01:42.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.139+0000 7faacf7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.139+0000 7f7b95573640 1 -- 10.20.193.90:0/3051108113 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b84003610 con 0x7f7b980ba350 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f5125ffb640 1 -- 10.20.193.90:0/1923812045 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2688393255 0 0) 0x7f5114003610 con 0x7f51280585e0 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f5125ffb640 1 -- 10.20.193.90:0/1923812045 >> v1:10.20.193.93:6789/0 conn(0x7f51280b9f80 legacy=0x7f5128163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f5125ffb640 1 -- 10.20.193.90:0/1923812045 >> v1:10.20.193.90:6789/0 conn(0x7f51280ba350 legacy=0x7f5128057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f5125ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f5125ffb640 1 monclient: found mon.noname-c 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f5125ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f5125ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f5125ffb640 1 -- 10.20.193.90:0/1923812045 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51280573f0 con 0x7f51280585e0 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f5125ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f5125ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f5125ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f5125ffb640 1 -- 10.20.193.90:0/1923812045 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5118003170 con 0x7f51280585e0 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f5125ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f5125ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f5125ffb640 10 monclient: dump: 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f512d9b3640 5 monclient: authenticate success, global_id 4415 2026-02-25T15:01:42.199 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f512d9b3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f1b7d65c640 1 -- 10.20.193.90:0/1408243345 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2006077572 0 0) 0x7f1b78163bb0 con 0x7f1b780ba350 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f1b7d65c640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f1b7d65c640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f1b7d65c640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f1b7d65c640 10 monclient(hunting): my global_id is 4438 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.140+0000 7f1b7d65c640 1 -- 10.20.193.90:0/1408243345 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b78163e90 con 0x7f1b780ba350 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.142+0000 7fee1ca3d640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fee18059e70 0x7fee18148e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54180/0 (socket says 10.20.193.90:54180) 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.142+0000 7fee1ca3d640 1 -- 10.20.193.90:0/1306701788 learned_addr learned my addr 10.20.193.90:0/1306701788 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.143+0000 7fee05ffb640 10 client.?.objecter ms_handle_connect 0x7fee18059e70 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.143+0000 7fee05ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.143+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1981877831 0 0) 0x7faaf012a4f0 con 0x7faaf0162f10 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.143+0000 7faacf7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.143+0000 7faacf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.143+0000 7faacf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.143+0000 7faacf7fe640 10 monclient(hunting): my global_id is 4430 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.143+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faaf0129f30 con 0x7faaf0162f10 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.144+0000 7fc0466e8640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fc0400b5da0 0x7fc04013b550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54166/0 (socket says 10.20.193.90:54166) 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.144+0000 7fc0466e8640 1 -- 10.20.193.90:0/3934146879 learned_addr learned my addr 10.20.193.90:0/3934146879 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.145+0000 7fee05ffb640 10 client.?.objecter ms_handle_connect 0x7fee1805a240 2026-02-25T15:01:42.200 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.145+0000 7fee05ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7fc02affd640 10 client.?.objecter ms_handle_connect 0x7fc0400b5da0 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7fc02affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f829f7fe640 1 -- 10.20.193.90:0/241044117 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2520483567 0 0) 0x7f82a81638a0 con 0x7f82a8159d10 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f829f7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f829f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f829f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f829f7fe640 10 monclient(hunting): my global_id is 4441 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f829f7fe640 1 -- 10.20.193.90:0/241044117 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8294003610 con 0x7f82a8159d10 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f73c3fff640 1 -- 10.20.193.90:0/81938051 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3079633846 0 0) 0x7f73d4163e90 con 0x7f73d40585e0 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f73c3fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f73c3fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f73c3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f73c3fff640 10 monclient(hunting): my global_id is 4421 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f73c3fff640 1 -- 10.20.193.90:0/81938051 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73c4002910 con 0x7f73d40585e0 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2241031819 0 0) 0x7f26a0003610 con 0x7f26d0164d00 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f26d012a110 con 0x7f26d0164d00 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f26cc0033d0 con 0x7f26d0164d00 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f26be7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f26be7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f26be7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.201 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7f26be7fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7fc84cca1640 1 -- 10.20.193.90:0/110573329 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3121444630 0 0) 0x7fc8480b3bd0 con 0x7fc8480b3190 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7fc84cca1640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7fc84cca1640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7fc84cca1640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.146+0000 7fc84cca1640 10 monclient(hunting): my global_id is 4427 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.147+0000 7fc84cca1640 1 -- 10.20.193.90:0/110573329 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc830003610 con 0x7fc8480b3190 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.147+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3863488957 0 0) 0x7f26d012a6d0 con 0x7f26d0157b50 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.147+0000 7f26be7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.147+0000 7f26be7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.147+0000 7f26be7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.147+0000 7f26be7fc640 10 monclient(hunting): my global_id is 4436 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.147+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26a0003610 con 0x7f26d0157b50 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f4f7affd640 1 -- 10.20.193.90:0/427355895 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4f68002fc0 con 0x7f4f7c15b220 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f4f7affd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f4f7affd640 1 -- 10.20.193.90:0/427355895 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4f68003400 con 0x7f4f7c15b220 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f4f7affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f4f7affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f4f7affd640 10 monclient: dump: 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f4f826b1640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f4f826b1640 4 set_mon_vals no callback set 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f0bfd30c640 1 -- 10.20.193.90:0/3350704169 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0bec003000 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f0bfd30c640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f0bfd30c640 1 -- 10.20.193.90:0/3350704169 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0bec003400 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f0c00599640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f0c00599640 4 set_mon_vals no callback set 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f4f826b1640 10 monclient: shutdown 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f4f826b1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f4f826b1640 1 -- 10.20.193.90:0/427355895 >> v1:10.20.193.93:6789/0 conn(0x7f4f7c15b220 legacy=0x7f4f7c058d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f0c00599640 10 monclient: shutdown 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f0bfd30c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.203 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f0bfd30c640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f0bfd30c640 10 monclient: dump: 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f0c00599640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f0c00599640 1 -- 10.20.193.90:0/3350704169 >> v1:10.20.193.93:6789/0 conn(0x7f0bf80b0ca0 legacy=0x7f0bf80b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f4f826b1640 1 -- 10.20.193.90:0/427355895 shutdown_connections 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f0c00599640 1 -- 10.20.193.90:0/3350704169 shutdown_connections 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f0c00599640 1 -- 10.20.193.90:0/3350704169 >> 10.20.193.90:0/3350704169 conn(0x7f0bf80b5290 msgr2=0x7f0bf80b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f4f826b1640 1 -- 10.20.193.90:0/427355895 >> 10.20.193.90:0/427355895 conn(0x7f4f7c0b4100 msgr2=0x7f4f7c0b4500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f4f826b1640 1 -- 10.20.193.90:0/427355895 shutdown_connections 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f0c00599640 1 -- 10.20.193.90:0/3350704169 shutdown_connections 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f4f826b1640 1 -- 10.20.193.90:0/427355895 wait complete. 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f0c00599640 1 -- 10.20.193.90:0/3350704169 wait complete. 2026-02-25T15:01:42.204 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f73c3fff640 1 -- 10.20.193.90:0/81938051 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2731534141 0 0) 0x7f73d4163bb0 con 0x7f73d40ba350 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f73c3fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f73c3fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f73c3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f73c3fff640 10 monclient(hunting): my global_id is 4435 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f5b95412640 1 -- 10.20.193.90:0/1899139546 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5b80003000 con 0x7f5b900b9f80 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f5b95412640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f5b95412640 1 -- 10.20.193.90:0/1899139546 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5b80003400 con 0x7f5b900b9f80 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f5b9869f640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f5b9869f640 4 set_mon_vals no callback set 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f5b95412640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f5b95412640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f5b95412640 10 monclient: dump: 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.148+0000 7f73c3fff640 1 -- 10.20.193.90:0/81938051 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73d4163e90 con 0x7f73d40ba350 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f7b95573640 1 -- 10.20.193.90:0/3051108113 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1751616429 0 0) 0x7f7b98163e90 con 0x7f7b980b9f80 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f7b95573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f7b95573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f7b95573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f7b95573640 10 monclient(hunting): my global_id is 4439 2026-02-25T15:01:42.205 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f7b95573640 1 -- 10.20.193.90:0/3051108113 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b98163bb0 con 0x7f7b980b9f80 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f5145ffb640 10 client.?.objecter ms_handle_connect 0x7f51580ba350 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f5145ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f4f826b1640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f4f826b1640 10 monclient: monmap: 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7fc84cca1640 1 -- 10.20.193.90:0/110573329 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 918341880 0 0) 0x7fc8480b38f0 con 0x7fc8480b5da0 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7fc84cca1640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7fc84cca1640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7fc84cca1640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7fc84cca1640 10 monclient(hunting): my global_id is 4444 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7fc84cca1640 1 -- 10.20.193.90:0/110573329 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8480b3bd0 con 0x7fc8480b5da0 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f5b9869f640 10 monclient: shutdown 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f5b9869f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f5b9869f640 1 -- 10.20.193.90:0/1899139546 >> v1:10.20.193.93:6789/0 conn(0x7f5b900b9f80 legacy=0x7f5b90163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.206 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f0c00599640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f2a8d7fa640 10 client.?.objecter ms_handle_connect 0x7f2a980b9f80 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f2a8d7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f0c00599640 10 monclient: monmap: 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f5b9869f640 1 -- 10.20.193.90:0/1899139546 shutdown_connections 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f2a8d7fa640 10 client.?.objecter ms_handle_connect 0x7f2a980ba350 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f2a8d7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f5b9869f640 1 -- 10.20.193.90:0/1899139546 >> 10.20.193.90:0/1899139546 conn(0x7f5b900b7990 msgr2=0x7f5b900b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f5b9869f640 1 -- 10.20.193.90:0/1899139546 shutdown_connections 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f5b9869f640 1 -- 10.20.193.90:0/1899139546 wait complete. 2026-02-25T15:01:42.207 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f4f826b1640 1 librados: starting msgr at 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f4f826b1640 1 librados: starting objecter 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f4f826b1640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f0c00599640 1 librados: starting msgr at 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f0c00599640 1 librados: starting objecter 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f4f826b1640 1 Processor -- start 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f0c00599640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f4f826b1640 1 -- start start 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f0c00599640 1 Processor -- start 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f4f826b1640 1 librados: setting wanted keys 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.149+0000 7f4f826b1640 1 librados: calling monclient init 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f4f826b1640 10 monclient: init 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1013553560 0 0) 0x7faaf012a210 con 0x7faaf0056c70 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7faacf7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7faacf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7faacf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7faacf7fe640 10 monclient(hunting): my global_id is 4447 2026-02-25T15:01:42.208 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faaf012a4f0 con 0x7faaf0056c70 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f5b9869f640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f5b9869f640 10 monclient: monmap: 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f0c00599640 1 -- start start 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f0c00599640 1 librados: setting wanted keys 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f0c00599640 1 librados: calling monclient init 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f0c00599640 10 monclient: init 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f5b9869f640 1 librados: starting msgr at 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f5b9869f640 1 librados: starting objecter 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f5b9869f640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f4f826b1640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f4f826b1640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f5b9869f640 1 Processor -- start 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f5b9869f640 1 -- start start 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f5b9869f640 1 librados: setting wanted keys 2026-02-25T15:01:42.209 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f5b9869f640 1 librados: calling monclient init 2026-02-25T15:01:42.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f5b9869f640 10 monclient: init 2026-02-25T15:01:42.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f4f826b1640 10 monclient(hunting): picked mon.noname-a con 0x7f4f7c055f60 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f4f826b1640 10 monclient(hunting): picked mon.noname-b con 0x7f4f7c059470 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f4f826b1640 10 monclient(hunting): picked mon.noname-c con 0x7f4f7c15b220 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f4f826b1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7c162950 con 0x7f4f7c055f60 2026-02-25T15:01:42.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f4f826b1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7c162c30 con 0x7f4f7c059470 2026-02-25T15:01:42.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f0c00599640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f4f826b1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7c162f10 con 0x7f4f7c15b220 2026-02-25T15:01:42.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f4f826b1640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f4f826b1640 10 monclient(hunting): authenticate will time out at 2991.750749s 2026-02-25T15:01:42.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f0c00599640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:42.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f0c00599640 10 monclient(hunting): picked mon.noname-b con 0x7f0bf80b0ca0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f0c00599640 10 monclient(hunting): picked mon.noname-a con 0x7f0bf80b28b0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f0c00599640 10 monclient(hunting): picked mon.noname-c con 0x7f0bf815d0c0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f0c00599640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bf8133fb0 con 0x7f0bf80b28b0 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f0c00599640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bf8134290 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f0c00599640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bf8134570 con 0x7f0bf815d0c0 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f0c00599640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f0c00599640 10 monclient(hunting): authenticate will time out at 2991.750959s 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f5b9869f640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f5b9869f640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f5b9869f640 10 monclient(hunting): picked mon.noname-b con 0x7f5b900585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f5b9869f640 10 monclient(hunting): picked mon.noname-a con 0x7f5b900b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f5b9869f640 10 monclient(hunting): picked mon.noname-c con 0x7f5b900ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f5b9869f640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b90161500 con 0x7f5b900b9f80 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f5b9869f640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b9015f760 con 0x7f5b900585e0 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f5b9869f640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b9015fa40 con 0x7f5b900ba350 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b95573640 1 -- 10.20.193.90:0/3051108113 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1120136034 0 0) 0x7f7b84003610 con 0x7f7b980ba350 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b95573640 1 -- 10.20.193.90:0/3051108113 >> v1:10.20.193.94:6789/0 conn(0x7f7b980b9f80 legacy=0x7f7b98163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b95573640 1 -- 10.20.193.90:0/3051108113 >> v1:10.20.193.90:6789/0 conn(0x7f7b980585e0 legacy=0x7f7b98166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b95573640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b95573640 1 monclient: found mon.noname-b 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b95573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b95573640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b95573640 1 -- 10.20.193.90:0/3051108113 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b980573f0 con 0x7f7b980ba350 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b95573640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b95573640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b95573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b95573640 1 -- 10.20.193.90:0/3051108113 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7b80002df0 con 0x7f7b980ba350 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b95573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b95573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b95573640 10 monclient: dump: 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.211 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b9cda2640 5 monclient: authenticate success, global_id 4432 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.150+0000 7f7b9cda2640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f5b9869f640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f5b9869f640 10 monclient(hunting): authenticate will time out at 2991.751026s 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73c3fff640 1 -- 10.20.193.90:0/81938051 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 809612834 0 0) 0x7f73bc003610 con 0x7f73d40b9f80 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73c3fff640 1 -- 10.20.193.90:0/81938051 >> v1:10.20.193.94:6789/0 conn(0x7f73d40585e0 legacy=0x7f73d4166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73c3fff640 1 -- 10.20.193.90:0/81938051 >> v1:10.20.193.93:6789/0 conn(0x7f73d40ba350 legacy=0x7f73d4057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73c3fff640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73c3fff640 1 monclient: found mon.noname-a 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73c3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73c3fff640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73c3fff640 1 -- 10.20.193.90:0/81938051 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d40573f0 con 0x7f73d40b9f80 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73c3fff640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73c3fff640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73c3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73c3fff640 1 -- 10.20.193.90:0/81938051 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f73c4002fc0 con 0x7f73d40b9f80 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73c3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73c3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73c3fff640 10 monclient: dump: 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.212 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73db755640 5 monclient: authenticate success, global_id 4407 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f73db755640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7fc02affd640 10 client.?.objecter ms_handle_connect 0x7fc040161640 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7fc02affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2980972843 0 0) 0x7f26d012a3f0 con 0x7f26d0157780 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f26d012a6d0 con 0x7f26d0157780 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f26c00030b0 con 0x7f26d0157780 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f26be7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f26be7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.151+0000 7f26be7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.152+0000 7fee05ffb640 10 client.?.objecter ms_handle_connect 0x7fee18058580 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.152+0000 7fee05ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.152+0000 7f47ca7fc640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f47cc0b9f80 0x7f47cc1fb060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44896/0 (socket says 10.20.193.90:44896) 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.152+0000 7f47ca7fc640 1 -- 10.20.193.90:0/2148629278 learned_addr learned my addr 10.20.193.90:0/2148629278 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.152+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1169747647 0 0) 0x7f2a98156bc0 con 0x7f2a980585e0 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.152+0000 7f2a8d7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.152+0000 7f2a8d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.152+0000 7f2a8d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.152+0000 7f2a8d7fa640 10 monclient(hunting): my global_id is 4425 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.152+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a6c003610 con 0x7f2a980585e0 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.152+0000 7f47ab7fe640 10 client.?.objecter ms_handle_connect 0x7f47cc0b9f80 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.152+0000 7f47ab7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b7d65c640 1 -- 10.20.193.90:0/1408243345 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1612183537 0 0) 0x7f1b68003610 con 0x7f1b780b9f80 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b7d65c640 1 -- 10.20.193.90:0/1408243345 >> v1:10.20.193.94:6789/0 conn(0x7f1b780585e0 legacy=0x7f1b78166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b7d65c640 1 -- 10.20.193.90:0/1408243345 >> v1:10.20.193.93:6789/0 conn(0x7f1b780ba350 legacy=0x7f1b78057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b7d65c640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b7d65c640 1 monclient: found mon.noname-a 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b7d65c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b7d65c640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b7d65c640 1 -- 10.20.193.90:0/1408243345 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b780573f0 con 0x7f1b780b9f80 2026-02-25T15:01:42.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b7d65c640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b7d65c640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b7d65c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b7d65c640 1 -- 10.20.193.90:0/1408243345 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1b70002fc0 con 0x7f1b780b9f80 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b7d65c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b7d65c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b7d65c640 10 monclient: dump: 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b808e9640 5 monclient: authenticate success, global_id 4413 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f1b808e9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1850162588 0 0) 0x7f5128003610 con 0x7f51580b9f80 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f51581fb730 con 0x7f51580b9f80 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f514c0031b0 con 0x7f51580b9f80 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f5145ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f5145ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f5145ffb640 10 monclient(hunting): dump: 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.153+0000 7f5145ffb640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.155+0000 7f47ab7fe640 10 client.?.objecter ms_handle_connect 0x7f47cc0585e0 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.155+0000 7f47ab7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.156+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3118834935 0 0) 0x7fee18135200 con 0x7fee18059e70 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.156+0000 7fee05ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.156+0000 7fee05ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.156+0000 7fee05ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.156+0000 7fee05ffb640 10 monclient(hunting): my global_id is 4442 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.156+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fede8003610 con 0x7fee18059e70 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.158+0000 7f47ab7fe640 10 client.?.objecter ms_handle_connect 0x7f47cc0ba350 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.158+0000 7f47ab7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.159+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3150404784 0 0) 0x7f5158159ea0 con 0x7f51580585e0 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.159+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5128003610 con 0x7f51580585e0 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.159+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5148003170 con 0x7f51580585e0 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.159+0000 7f5145ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.159+0000 7f5145ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.159+0000 7f5145ffb640 10 monclient(hunting): dump: 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.216 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.159+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2423660178 0 0) 0x7f2a981ff450 con 0x7f2a980ba350 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.159+0000 7f2a8d7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.159+0000 7f2a8d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.159+0000 7f2a8d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.159+0000 7f2a8d7fa640 10 monclient(hunting): my global_id is 4453 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.159+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a98156bc0 con 0x7f2a980ba350 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f829f7fe640 1 -- 10.20.193.90:0/241044117 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4218030930 0 0) 0x7f8294003610 con 0x7f82a8159d10 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f829f7fe640 1 -- 10.20.193.90:0/241044117 >> v1:10.20.193.94:6789/0 conn(0x7f82a805a310 legacy=0x7f82a81c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f829f7fe640 1 -- 10.20.193.90:0/241044117 >> v1:10.20.193.90:6789/0 conn(0x7f82a805d990 legacy=0x7f82a8059c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f829f7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f829f7fe640 1 monclient: found mon.noname-b 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f829f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f829f7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f829f7fe640 1 -- 10.20.193.90:0/241044117 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a8163e60 con 0x7f82a8159d10 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f829f7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f829f7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f829f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f829f7fe640 1 -- 10.20.193.90:0/241044117 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8298002df0 con 0x7f82a8159d10 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f82aecd0640 5 monclient: authenticate success, global_id 4441 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f82aecd0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f829f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f829f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.217 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f829f7fe640 10 monclient: dump: 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f82aecd0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1985778519 0 0) 0x7faabc003610 con 0x7faaf0155b20 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faaf012a210 con 0x7faaf0155b20 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7faad80030f0 con 0x7faaf0155b20 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7faacf7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7faacf7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7faacf7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.218 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7faacf7fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f5125ffb640 1 -- 10.20.193.90:0/1923812045 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5118003380 con 0x7f51280585e0 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f5125ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f5125ffb640 1 -- 10.20.193.90:0/1923812045 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f51180037c0 con 0x7f51280585e0 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f5125ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f5125ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f5125ffb640 10 monclient: dump: 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f512d9b3640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.160+0000 7f512d9b3640 4 set_mon_vals no callback set 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.161+0000 7f512d9b3640 10 monclient: shutdown 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.161+0000 7f512d9b3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.161+0000 7f512d9b3640 1 -- 10.20.193.90:0/1923812045 >> v1:10.20.193.94:6789/0 conn(0x7f51280585e0 legacy=0x7f5128166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.161+0000 7f512d9b3640 1 -- 10.20.193.90:0/1923812045 shutdown_connections 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.161+0000 7f512d9b3640 1 -- 10.20.193.90:0/1923812045 >> 10.20.193.90:0/1923812045 conn(0x7f51280b7990 msgr2=0x7f51280b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.161+0000 7f512d9b3640 1 -- 10.20.193.90:0/1923812045 shutdown_connections 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.161+0000 7f512d9b3640 1 -- 10.20.193.90:0/1923812045 wait complete. 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f512d9b3640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f512d9b3640 10 monclient: monmap: 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f512d9b3640 1 librados: starting msgr at 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f512d9b3640 1 librados: starting objecter 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f512d9b3640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f512d9b3640 1 Processor -- start 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 44296562 0 0) 0x7f51581fb450 con 0x7f51580ba350 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f5145ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f5145ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f5145ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f5145ffb640 10 monclient(hunting): my global_id is 4450 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5158159ea0 con 0x7f51580ba350 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f512d9b3640 1 -- start start 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f512d9b3640 1 librados: setting wanted keys 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f512d9b3640 1 librados: calling monclient init 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f512d9b3640 10 monclient: init 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.162+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4288395302 0 0) 0x7f26d012a110 con 0x7f26d0164d00 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 >> v1:10.20.193.94:6789/0 conn(0x7f26d0157b50 legacy=0x7f26d013ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 >> v1:10.20.193.90:6789/0 conn(0x7f26d0157780 legacy=0x7f26d013ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26be7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26be7fc640 1 monclient: found mon.b 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3312422477 0 0) 0x7fee18134f20 con 0x7fee18058580 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fee05ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fee05ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fee05ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26be7fc640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d012a9b0 con 0x7f26d0164d00 2026-02-25T15:01:42.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26be7fc640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26be7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26d7c5b640 5 monclient: authenticate success, global_id 4429 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26d7c5b640 10 monclient: _renew_subs 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fee05ffb640 10 monclient(hunting): my global_id is 4459 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee18135200 con 0x7fee18058580 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26d7c5b640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26d7c5b640 1 -- 10.20.193.90:0/3380070399 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f26d0129d00 con 0x7f26d0164d00 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f512d9b3640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f512d9b3640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26d7c5b640 10 client.4429.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26d7c5b640 10 monclient: _renew_subs 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26d7c5b640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26d7c5b640 1 -- 10.20.193.90:0/3380070399 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f26d013d720 con 0x7f26d0164d00 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26d7c5b640 1 librados: init done 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f512d9b3640 10 monclient(hunting): picked mon.noname-a con 0x7f51280585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f512d9b3640 10 monclient(hunting): picked mon.noname-b con 0x7f51280b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f512d9b3640 10 monclient(hunting): picked mon.noname-c con 0x7f51280ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f512d9b3640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51280b7060 con 0x7f51280585e0 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f512d9b3640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51281614a0 con 0x7f51280b9f80 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f512d9b3640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5128161780 con 0x7f51280ba350 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f512d9b3640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f512d9b3640 10 monclient(hunting): authenticate will time out at 2991.763144s 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f4f7bfff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f4f7c055f60 0x7f4f7c150a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44904/0 (socket says 10.20.193.90:44904) 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f4f7bfff640 1 -- 10.20.193.90:0/429045463 learned_addr learned my addr 10.20.193.90:0/429045463 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1657306870 0 0) 0x7fee18134c40 con 0x7fee1805a240 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fee05ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fee05ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fee05ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fee05ffb640 10 monclient(hunting): my global_id is 4428 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee18134f20 con 0x7fee1805a240 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26d7c5b640 10 client.4429.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26d7c5b640 15 client.4429.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26d7c5b640 20 client.4429.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26d7c5b640 10 client.4429.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26d7c5b640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7f26d7c5b640 1 -- 10.20.193.90:0/3380070399 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f26d00531e0 con 0x7f26d0164d00 2026-02-25T15:01:42.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84cca1640 1 -- 10.20.193.90:0/110573329 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4289604837 0 0) 0x7fc830003610 con 0x7fc8480b3190 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84cca1640 1 -- 10.20.193.90:0/110573329 >> v1:10.20.193.93:6789/0 conn(0x7fc8480b5da0 legacy=0x7fc8480b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84cca1640 1 -- 10.20.193.90:0/110573329 >> v1:10.20.193.90:6789/0 conn(0x7fc84815b0a0 legacy=0x7fc848162300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84cca1640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84cca1640 1 monclient: found mon.noname-c 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84cca1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84cca1640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84cca1640 1 -- 10.20.193.90:0/110573329 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8480b3eb0 con 0x7fc8480b3190 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84cca1640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84cca1640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84cca1640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84cca1640 1 -- 10.20.193.90:0/110573329 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fc83c002df0 con 0x7fc8480b3190 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84cca1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84cca1640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84cca1640 10 monclient: dump: 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84ff2e640 5 monclient: authenticate success, global_id 4427 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.163+0000 7fc84ff2e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7fc02affd640 10 client.?.objecter ms_handle_connect 0x7fc040161a10 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7fc02affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 947631900 0 0) 0x7f47cc1fbcb0 con 0x7f47cc0b9f80 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f47ab7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f47ab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f47ab7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f47ab7fe640 10 monclient(hunting): my global_id is 4434 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4798003610 con 0x7f47cc0b9f80 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f4f78ff9640 10 client.?.objecter ms_handle_connect 0x7f4f7c055f60 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f4f78ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f73c3fff640 1 -- 10.20.193.90:0/81938051 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f73c4003190 con 0x7f73d40b9f80 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f73c3fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f73c3fff640 1 -- 10.20.193.90:0/81938051 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f73c4004990 con 0x7f73d40b9f80 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f73c3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f73c3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f73c3fff640 10 monclient: dump: 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f73db755640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.164+0000 7f73db755640 4 set_mon_vals no callback set 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7f73db755640 10 monclient: shutdown 2026-02-25T15:01:42.224 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7f73db755640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7f73db755640 1 -- 10.20.193.90:0/81938051 >> v1:10.20.193.90:6789/0 conn(0x7f73d40b9f80 legacy=0x7f73d4163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7f73db755640 1 -- 10.20.193.90:0/81938051 shutdown_connections 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7f73db755640 1 -- 10.20.193.90:0/81938051 >> 10.20.193.90:0/81938051 conn(0x7f73d40b7990 msgr2=0x7f73d40b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7f73db755640 1 -- 10.20.193.90:0/81938051 shutdown_connections 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7f73db755640 1 -- 10.20.193.90:0/81938051 wait complete. 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7f73db755640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7f73db755640 10 monclient: monmap: 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2670344197 0 0) 0x7fc04013a790 con 0x7fc040161640 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7fc02affd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7fc02affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7fc02affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7fc02affd640 10 monclient(hunting): my global_id is 4456 2026-02-25T15:01:42.225 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc010003610 con 0x7fc040161640 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7f73db755640 1 librados: starting msgr at 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7f73db755640 1 librados: starting objecter 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7f73db755640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.165+0000 7f73db755640 1 Processor -- start 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f73db755640 1 -- start start 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f73db755640 1 librados: setting wanted keys 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f73db755640 1 librados: calling monclient init 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f73db755640 10 monclient: init 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f73db755640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f73db755640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f73db755640 10 monclient(hunting): picked mon.noname-a con 0x7f73d40585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f73db755640 10 monclient(hunting): picked mon.noname-b con 0x7f73d40b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f73db755640 10 monclient(hunting): picked mon.noname-c con 0x7f73d40ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f73db755640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d40b4ea0 con 0x7f73d40585e0 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f73db755640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d40b23b0 con 0x7f73d40b9f80 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f73db755640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d40b2690 con 0x7f73d40ba350 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f73db755640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f73db755640 10 monclient(hunting): authenticate will time out at 2991.766516s 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f5b95c13640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f5b900b9f80 0x7f5b901564b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44932/0 (socket says 10.20.193.90:44932) 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.166+0000 7f5b95c13640 1 -- 10.20.193.90:0/2131604963 learned_addr learned my addr 10.20.193.90:0/2131604963 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.170+0000 7f5b76ffd640 10 client.?.objecter ms_handle_connect 0x7f5b900b9f80 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.170+0000 7f5b76ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.170+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4170024672 0 0) 0x7f47cc162880 con 0x7f47cc0585e0 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.170+0000 7f47ab7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.170+0000 7f47ab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.170+0000 7f47ab7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.170+0000 7f47ab7fe640 10 monclient(hunting): my global_id is 4462 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.170+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47cc1fbcb0 con 0x7f47cc0585e0 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.170+0000 7f0bfdb0d640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f0bf80b28b0 0x7f0bf81563f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:44916/0 (socket says 10.20.193.90:44916) 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.170+0000 7f0bfdb0d640 1 -- 10.20.193.90:0/1950449469 learned_addr learned my addr 10.20.193.90:0/1950449469 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1418313018 0 0) 0x7fc04013aa70 con 0x7fc0400b5da0 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7fc02affd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7fc02affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7fc02affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7fc02affd640 10 monclient(hunting): my global_id is 4448 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc04013a790 con 0x7fc0400b5da0 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7f7b95573640 1 -- 10.20.193.90:0/3051108113 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7b80003000 con 0x7f7b980ba350 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7f7b95573640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7f7b95573640 1 -- 10.20.193.90:0/3051108113 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7b80003440 con 0x7f7b980ba350 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7f7b95573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7f7b95573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7f7b95573640 10 monclient: dump: 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7f7b9cda2640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7f7b9cda2640 4 set_mon_vals no callback set 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7f0bdeffd640 10 client.?.objecter ms_handle_connect 0x7f0bf80b28b0 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.171+0000 7f0bdeffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 10 monclient: shutdown 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 1 -- 10.20.193.90:0/3051108113 >> v1:10.20.193.93:6789/0 conn(0x7f7b980ba350 legacy=0x7f7b98057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 1 -- 10.20.193.90:0/3051108113 shutdown_connections 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 1 -- 10.20.193.90:0/3051108113 >> 10.20.193.90:0/3051108113 conn(0x7f7b980b7990 msgr2=0x7f7b980b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 1 -- 10.20.193.90:0/3051108113 shutdown_connections 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 1 -- 10.20.193.90:0/3051108113 wait complete. 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 10 monclient: monmap: 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1975080900 0 0) 0x7fede8003610 con 0x7fee18059e70 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fee18134c40 con 0x7fee18059e70 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fee0c0033d0 con 0x7fee18059e70 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7fee05ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7fee05ffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7fee05ffb640 10 monclient(hunting): dump: 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7fee05ffb640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 1 librados: starting msgr at 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 1 librados: starting objecter 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 1 Processor -- start 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 1 -- start start 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 1 librados: setting wanted keys 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 1 librados: calling monclient init 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.172+0000 7f7b9cda2640 10 monclient: init 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3884822455 0 0) 0x7f2a981fb140 con 0x7f2a980b9f80 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f2a8d7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f2a8d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f2a8d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f2a8d7fa640 10 monclient(hunting): my global_id is 4451 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a981ff450 con 0x7f2a980b9f80 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1471239492 0 0) 0x7f47cc162b60 con 0x7f47cc0ba350 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f47ab7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f47ab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f47ab7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f47ab7fe640 10 monclient(hunting): my global_id is 4454 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47cc162880 con 0x7f47cc0ba350 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f51277fe640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f51280ba350 0x7f51281559f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54228/0 (socket says 10.20.193.90:54228) 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f51277fe640 1 -- 10.20.193.90:0/1178662605 learned_addr learned my addr 10.20.193.90:0/1178662605 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f7b9cda2640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f7b9cda2640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f7b9cda2640 10 monclient(hunting): picked mon.noname-a con 0x7f7b980585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f7b9cda2640 10 monclient(hunting): picked mon.noname-c con 0x7f7b980b9f80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f7b9cda2640 10 monclient(hunting): picked mon.noname-b con 0x7f7b980ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f7b9cda2640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b981ff3d0 con 0x7f7b980585e0 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f7b9cda2640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b980afc40 con 0x7f7b980ba350 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f7b9cda2640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b980b1970 con 0x7f7b980b9f80 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f7b9cda2640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f7b9cda2640 10 monclient(hunting): authenticate will time out at 2991.773448s 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f4f78ff9640 10 client.?.objecter ms_handle_connect 0x7f4f7c15b220 2026-02-25T15:01:42.230 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f4f78ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f50fffff640 10 client.?.objecter ms_handle_connect 0x7f51280ba350 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f50fffff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f5b76ffd640 10 client.?.objecter ms_handle_connect 0x7f5b900ba350 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.173+0000 7f5b76ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2354509524 0 0) 0x7faaf012a210 con 0x7faaf0155b20 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 >> v1:10.20.193.94:6789/0 conn(0x7faaf0162f10 legacy=0x7faaf0145510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 >> v1:10.20.193.93:6789/0 conn(0x7faaf0056c70 legacy=0x7faaf012b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faacf7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faacf7fe640 1 monclient: found mon.a 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faacf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faacf7fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf012a7d0 con 0x7faaf0155b20 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faacf7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faacf7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faacf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faaf52d1640 5 monclient: authenticate success, global_id 4422 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faaf52d1640 10 monclient: _renew_subs 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faaf52d1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faaf52d1640 1 -- 10.20.193.90:0/2340633913 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7faaf012aab0 con 0x7faaf0155b20 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faaf52d1640 10 client.4422.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faaf52d1640 10 monclient: _renew_subs 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faaf52d1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faaf52d1640 1 -- 10.20.193.90:0/2340633913 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7faaf015c7f0 con 0x7faaf0155b20 2026-02-25T15:01:42.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faaf52d1640 1 librados: init done 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3890104658 0 0) 0x7f2a6c003610 con 0x7f2a980585e0 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2a981fb140 con 0x7f2a980585e0 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2a88003350 con 0x7f2a980585e0 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7f2a8d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7f2a8d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7f2a8d7fa640 10 monclient(hunting): dump: 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7f2a8d7fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faaf52d1640 10 client.4422.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faaf52d1640 15 client.4422.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faaf52d1640 20 client.4422.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faaf52d1640 10 client.4422.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faaf52d1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.174+0000 7faaf52d1640 1 -- 10.20.193.90:0/2340633913 --> v1:10.20.193.90:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7faaf00531e0 con 0x7faaf0155b20 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4137884226 0 0) 0x7fee18134f20 con 0x7fee1805a240 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fede8003610 con 0x7fee1805a240 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fee14003370 con 0x7fee1805a240 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7fee05ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7fee05ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7fee05ffb640 10 monclient(hunting): dump: 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.232 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f1b7d65c640 1 -- 10.20.193.90:0/1408243345 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1b70003190 con 0x7f1b780b9f80 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f1b7d65c640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f1b7d65c640 1 -- 10.20.193.90:0/1408243345 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1b700035d0 con 0x7f1b780b9f80 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f1b7d65c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f1b7d65c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f1b7d65c640 10 monclient: dump: 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f1b808e9640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f1b808e9640 4 set_mon_vals no callback set 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f0bdeffd640 10 client.?.objecter ms_handle_connect 0x7f0bf815d0c0 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f0bdeffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 900739243 0 0) 0x7f5128003610 con 0x7f51580585e0 2026-02-25T15:01:42.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 >> v1:10.20.193.93:6789/0 conn(0x7f51580ba350 legacy=0x7f5158160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 >> v1:10.20.193.90:6789/0 conn(0x7f51580b9f80 legacy=0x7f515815daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f5145ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f5145ffb640 1 monclient: found mon.c 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f5145ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f5145ffb640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51581fba10 con 0x7f51580585e0 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f5145ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f5145ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f5145ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f515f98a640 5 monclient: authenticate success, global_id 4433 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f515f98a640 10 monclient: _renew_subs 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f515f98a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f515f98a640 1 -- 10.20.193.90:0/2774738662 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f51581fbcf0 con 0x7f51580585e0 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f1b808e9640 10 monclient: shutdown 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f1b808e9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f1b808e9640 1 -- 10.20.193.90:0/1408243345 >> v1:10.20.193.90:6789/0 conn(0x7f1b780b9f80 legacy=0x7f1b78163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f515f98a640 10 client.4433.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f515f98a640 10 monclient: _renew_subs 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f515f98a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f515f98a640 1 -- 10.20.193.90:0/2774738662 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f515815faf0 con 0x7f51580585e0 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f515f98a640 1 librados: init done 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f1b808e9640 1 -- 10.20.193.90:0/1408243345 shutdown_connections 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f1b808e9640 1 -- 10.20.193.90:0/1408243345 >> 10.20.193.90:0/1408243345 conn(0x7f1b780b7990 msgr2=0x7f1b780b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f1b808e9640 1 -- 10.20.193.90:0/1408243345 shutdown_connections 2026-02-25T15:01:42.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.175+0000 7f1b808e9640 1 -- 10.20.193.90:0/1408243345 wait complete. 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1166766312 0 0) 0x7f4798003610 con 0x7f47cc0b9f80 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f47cc162b60 con 0x7f47cc0b9f80 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f47b4004030 con 0x7f47cc0b9f80 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f47ab7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f47ab7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f47ab7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f47ab7fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2670531516 0 0) 0x7fc04013a4b0 con 0x7fc040161a10 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7fc02affd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7fc02affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7fc02affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7fc02affd640 10 monclient(hunting): my global_id is 4440 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc04013aa70 con 0x7fc040161a10 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f515f98a640 10 client.4433.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f515f98a640 15 client.4433.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f515f98a640 20 client.4433.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f515f98a640 10 client.4433.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f515f98a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f515f98a640 1 -- 10.20.193.90:0/2774738662 --> v1:10.20.193.94:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f515815e210 con 0x7f51580585e0 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f1b808e9640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f1b808e9640 10 monclient: monmap: 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f1b808e9640 1 librados: starting msgr at 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f1b808e9640 1 librados: starting objecter 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f1b808e9640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f1b808e9640 1 Processor -- start 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f1b808e9640 1 -- start start 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f1b808e9640 1 librados: setting wanted keys 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f1b808e9640 1 librados: calling monclient init 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.176+0000 7f1b808e9640 10 monclient: init 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.177+0000 7f1b808e9640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.177+0000 7f1b808e9640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.177+0000 7f1b808e9640 10 monclient(hunting): picked mon.noname-c con 0x7f1b780585e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.177+0000 7f1b808e9640 10 monclient(hunting): picked mon.noname-b con 0x7f1b780b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.177+0000 7f1b808e9640 10 monclient(hunting): picked mon.noname-a con 0x7f1b780ba350 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.177+0000 7f1b808e9640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b780b4ea0 con 0x7f1b780ba350 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.177+0000 7f1b808e9640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b780b23b0 con 0x7f1b780b9f80 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.177+0000 7f1b808e9640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b780b2690 con 0x7f1b780585e0 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.177+0000 7f1b808e9640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.177+0000 7f1b808e9640 10 monclient(hunting): authenticate will time out at 2991.777275s 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.177+0000 7f0bdeffd640 10 client.?.objecter ms_handle_connect 0x7f0bf80b0ca0 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.177+0000 7f0bdeffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.178+0000 7f4f78ff9640 10 client.?.objecter ms_handle_connect 0x7f4f7c059470 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.178+0000 7f4f78ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.179+0000 7f5b76ffd640 10 client.?.objecter ms_handle_connect 0x7f5b900585e0 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.179+0000 7f5b76ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.179+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4064468065 0 0) 0x7fee18135200 con 0x7fee18058580 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.179+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fee18134f20 con 0x7fee18058580 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.179+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fee080030b0 con 0x7fee18058580 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.179+0000 7fee05ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.179+0000 7fee05ffb640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.179+0000 7fee05ffb640 10 monclient(hunting): dump: 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.237 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.180+0000 7fc84cca1640 1 -- 10.20.193.90:0/110573329 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc83c003000 con 0x7fc8480b3190 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.180+0000 7fc84cca1640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.180+0000 7fc84cca1640 1 -- 10.20.193.90:0/110573329 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fc83c003440 con 0x7fc8480b3190 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.180+0000 7fc84cca1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.180+0000 7fc84cca1640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.180+0000 7fc84cca1640 10 monclient: dump: 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.180+0000 7fc84ff2e640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.180+0000 7fc84ff2e640 4 set_mon_vals no callback set 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.181+0000 7fc84ff2e640 10 monclient: shutdown 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.181+0000 7fc84ff2e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.181+0000 7fc84ff2e640 1 -- 10.20.193.90:0/110573329 >> v1:10.20.193.94:6789/0 conn(0x7fc8480b3190 legacy=0x7fc8480b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.181+0000 7fc84ff2e640 1 -- 10.20.193.90:0/110573329 shutdown_connections 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.181+0000 7fc84ff2e640 1 -- 10.20.193.90:0/110573329 >> 10.20.193.90:0/110573329 conn(0x7fc8480b5290 msgr2=0x7fc8480b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.181+0000 7fc84ff2e640 1 -- 10.20.193.90:0/110573329 shutdown_connections 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.181+0000 7fc84ff2e640 1 -- 10.20.193.90:0/110573329 wait complete. 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.181+0000 7fc84ff2e640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.181+0000 7f50fffff640 10 client.?.objecter ms_handle_connect 0x7f51280b9f80 2026-02-25T15:01:42.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.181+0000 7f50fffff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.181+0000 7fc84ff2e640 10 monclient: monmap: 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.181+0000 7fc84ff2e640 1 librados: starting msgr at 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.181+0000 7fc84ff2e640 1 librados: starting objecter 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.181+0000 7fc84ff2e640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.182+0000 7fc84ff2e640 1 Processor -- start 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.182+0000 7fc84ff2e640 1 -- start start 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.182+0000 7fc84ff2e640 1 librados: setting wanted keys 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.182+0000 7fc84ff2e640 1 librados: calling monclient init 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.182+0000 7fc84ff2e640 10 monclient: init 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.182+0000 7fc84ff2e640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.182+0000 7fc84ff2e640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.182+0000 7fc84ff2e640 10 monclient(hunting): picked mon.noname-a con 0x7fc8480b5da0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.182+0000 7fc84ff2e640 10 monclient(hunting): picked mon.noname-c con 0x7fc84815b0a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.182+0000 7fc84ff2e640 10 monclient(hunting): picked mon.noname-b con 0x7fc848131f20 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.182+0000 7fc84ff2e640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc848134ef0 con 0x7fc8480b5da0 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.182+0000 7fc84ff2e640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc848160720 con 0x7fc848131f20 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.182+0000 7fc84ff2e640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc848133b70 con 0x7fc84815b0a0 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.182+0000 7fc84ff2e640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.182+0000 7fc84ff2e640 10 monclient(hunting): authenticate will time out at 2991.782527s 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.183+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3403265126 0 0) 0x7f0bf8133fb0 con 0x7f0bf80b28b0 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.183+0000 7f0bdeffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.183+0000 7f0bdeffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.183+0000 7f0bdeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.183+0000 7f0bdeffd640 10 monclient(hunting): my global_id is 4449 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.183+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bc8003610 con 0x7f0bf80b28b0 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.184+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2362617232 0 0) 0x7f4f7c162f10 con 0x7f4f7c15b220 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.184+0000 7f4f78ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.184+0000 7f4f78ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.184+0000 7f4f78ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.184+0000 7f4f78ff9640 10 monclient(hunting): my global_id is 4457 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.184+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f4c003610 con 0x7f4f7c15b220 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.184+0000 7f73d9ccb640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f73d40ba350 0x7f73d41fe480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54230/0 (socket says 10.20.193.90:54230) 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.184+0000 7f73d9ccb640 1 -- 10.20.193.90:0/1294973177 learned_addr learned my addr 10.20.193.90:0/1294973177 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.184+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1714246573 0 0) 0x7f5b9015fa40 con 0x7f5b900ba350 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.184+0000 7f5b76ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.184+0000 7f5b76ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.184+0000 7f5b76ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.184+0000 7f5b76ffd640 10 monclient(hunting): my global_id is 4463 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.184+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b60003610 con 0x7f5b900ba350 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f73c1ffb640 10 client.?.objecter ms_handle_connect 0x7f73d40ba350 2026-02-25T15:01:42.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f73c1ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2369966090 0 0) 0x7fee18134c40 con 0x7fee18059e70 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 >> v1:10.20.193.93:6789/0 conn(0x7fee18058580 legacy=0x7fee18138ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 >> v1:10.20.193.90:6789/0 conn(0x7fee1805a240 legacy=0x7fee18133e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee05ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee05ffb640 1 monclient: found mon.c 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee05ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee05ffb640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee18137d10 con 0x7fee18059e70 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee05ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee05ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee05ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee1f4c9640 5 monclient: authenticate success, global_id 4442 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee1f4c9640 10 monclient: _renew_subs 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee1f4c9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2902675515 0 0) 0x7f0bf8134290 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee1f4c9640 1 -- 10.20.193.90:0/1306701788 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fee18137ff0 con 0x7fee18059e70 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f0bdeffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f0bdeffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f0bdeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f0bdeffd640 10 monclient(hunting): my global_id is 4465 2026-02-25T15:01:42.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bf8133fb0 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee1f4c9640 10 client.4442.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee1f4c9640 10 monclient: _renew_subs 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee1f4c9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee1f4c9640 1 -- 10.20.193.90:0/1306701788 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fee181385b0 con 0x7fee18059e70 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee1f4c9640 1 librados: init done 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4158131246 0 0) 0x7f2a98156bc0 con 0x7f2a980ba350 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2a6c003610 con 0x7f2a980ba350 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2a94003360 con 0x7f2a980ba350 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f2a8d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f2a8d7fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f2a8d7fa640 10 monclient(hunting): dump: 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee1f4c9640 10 client.4442.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee1f4c9640 15 client.4442.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee1f4c9640 20 client.4442.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee1f4c9640 10 client.4442.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee1f4c9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7fee1f4c9640 1 -- 10.20.193.90:0/1306701788 --> v1:10.20.193.94:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fee18060200 con 0x7fee18059e70 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f829f7fe640 1 -- 10.20.193.90:0/241044117 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8298003070 con 0x7f82a8159d10 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f829f7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f829f7fe640 1 -- 10.20.193.90:0/241044117 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8298003470 con 0x7f82a8159d10 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f829f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f829f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f829f7fe640 10 monclient: dump: 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f82aecd0640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.185+0000 7f82aecd0640 4 set_mon_vals no callback set 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.186+0000 7f82aecd0640 10 monclient: shutdown 2026-02-25T15:01:42.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.186+0000 7f82aecd0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.186+0000 7f82aecd0640 1 -- 10.20.193.90:0/241044117 >> v1:10.20.193.93:6789/0 conn(0x7f82a8159d10 legacy=0x7f82a805d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.186+0000 7f82aecd0640 1 -- 10.20.193.90:0/241044117 shutdown_connections 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.186+0000 7f82aecd0640 1 -- 10.20.193.90:0/241044117 >> 10.20.193.90:0/241044117 conn(0x7f82a80590f0 msgr2=0x7f82a80594f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.186+0000 7f82aecd0640 1 -- 10.20.193.90:0/241044117 shutdown_connections 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.186+0000 7f82aecd0640 1 -- 10.20.193.90:0/241044117 wait complete. 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 10 monclient: monmap: 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 1 librados: starting msgr at 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 1 librados: starting objecter 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 1 Processor -- start 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 1 -- start start 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 1 librados: setting wanted keys 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 1 librados: calling monclient init 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 10 monclient: init 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 10 monclient(hunting): picked mon.noname-b con 0x7f82a805a310 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 10 monclient(hunting): picked mon.noname-a con 0x7f82a805d990 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 10 monclient(hunting): picked mon.noname-c con 0x7f82a8159d10 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82a811ebc0 con 0x7f82a805d990 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82a80b0210 con 0x7f82a805a310 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82a80b04f0 con 0x7f82a8159d10 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.187+0000 7f82aecd0640 10 monclient(hunting): authenticate will time out at 2991.787895s 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.189+0000 7f50fffff640 10 client.?.objecter ms_handle_connect 0x7f51280585e0 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.189+0000 7f50fffff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.190+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2166816391 0 0) 0x7f5128161780 con 0x7f51280ba350 2026-02-25T15:01:42.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.190+0000 7f50fffff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.190+0000 7f50fffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.190+0000 7f50fffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.190+0000 7f50fffff640 10 monclient(hunting): my global_id is 4460 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.190+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50f8003610 con 0x7f51280ba350 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.191+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3416217298 0 0) 0x7fc04013a790 con 0x7fc0400b5da0 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.191+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc04013a4b0 con 0x7fc0400b5da0 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.191+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fc0380030c0 con 0x7fc0400b5da0 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.191+0000 7fc02affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.191+0000 7fc02affd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.191+0000 7fc02affd640 10 monclient(hunting): dump: 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.191+0000 7fc02affd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.193+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f26d00531e0 con 0x7f26d0164d00 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.193+0000 7f26be7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.193+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f26cc003c20 con 0x7f26d0164d00 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.193+0000 7f26be7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.193+0000 7f26be7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.193+0000 7f26be7fc640 10 monclient: dump: 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.245 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.193+0000 7f26bffff640 4 set_mon_vals no callback set 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7faaf00531e0 con 0x7faaf0155b20 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7faacf7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7faad80039b0 con 0x7faaf0155b20 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7faacf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7faacf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7faacf7fe640 10 monclient: dump: 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7faaed7fa640 4 set_mon_vals no callback set 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 19870059 0 0) 0x7f2a981ff450 con 0x7f2a980b9f80 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2a98156bc0 con 0x7f2a980b9f80 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2a80002fb0 con 0x7f2a980b9f80 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7f2a8d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7f2a8d7fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7f2a8d7fa640 10 monclient(hunting): dump: 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.246 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7f7b95d74640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f7b980b9f80 0x7f7b9815af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54246/0 (socket says 10.20.193.90:54246) 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7f7b95d74640 1 -- 10.20.193.90:0/174363358 learned_addr learned my addr 10.20.193.90:0/174363358 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f26cc003e00 con 0x7f26d0164d00 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7faad8003bc0 con 0x7faaf0155b20 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7f26d59d0640 10 monclient: get_auth_request con 0x7f26a0079fe0 auth_method 0 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.194+0000 7f26be7fc640 10 client.4429.objecter ms_handle_connect 0x7f26a0079fe0 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7faaeeffd640 10 monclient: get_auth_request con 0x7faabc079fb0 auth_method 0 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f7b7effd640 10 client.?.objecter ms_handle_connect 0x7f7b980b9f80 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f7b7effd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7faacf7fe640 10 client.4422.objecter ms_handle_connect 0x7faabc079fb0 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f26cc095d50 con 0x7f26d0164d00 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 10 client.4429.objecter ms_dispatch2 0x7f26d0000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 3 client.4429.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 3 client.4429.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 10 client.4429.objecter checking command 1 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 20 client.4429.objecter _get_session s=0x7f26a00a2ec0 osd=1 3 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 20 client.4429.objecter put_session s=0x7f26a00a2ec0 osd=1 3 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 15 client.4429.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 20 client.4429.objecter _get_session s=0x7f26a00a2ec0 osd=1 3 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 20 client.4429.objecter get_session s=0x7f26a00a2ec0 osd=1 3 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 15 client.4429.objecter _session_command_op_assign 1 1 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 20 client.4429.objecter put_session s=0x7f26a00a2ec0 osd=1 4 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 10 client.4429.objecter _send_command 1 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 --> v1:10.20.193.93:6804/3161035071 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f26a00c5170 con 0x7f26a00a3570 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 20 client.4429.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2799705323 0 0) 0x7f5b9015f760 con 0x7f5b900585e0 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f5b76ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f5b76ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f5b76ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f5b76ffd640 10 monclient(hunting): my global_id is 4471 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b9015fa40 con 0x7f5b900585e0 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7f26cc05b320 con 0x7f26d0164d00 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.195+0000 7f26be7fc640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.196+0000 7f1b7de5d640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f1b780b9f80 0x7f1b781fc380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46552/0 (socket says 10.20.193.90:46552) 2026-02-25T15:01:42.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.196+0000 7f1b7de5d640 1 -- 10.20.193.90:0/2348609463 learned_addr learned my addr 10.20.193.90:0/2348609463 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.196+0000 7f73c1ffb640 10 client.?.objecter ms_handle_connect 0x7f73d40b9f80 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.196+0000 7f73c1ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.196+0000 7f26d51cf640 10 monclient: get_auth_request con 0x7f26a00a3570 auth_method 0 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.196+0000 7f1b62ffd640 10 client.?.objecter ms_handle_connect 0x7f1b780b9f80 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.196+0000 7f1b62ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7faad8095a90 con 0x7faaf0155b20 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 10 client.4422.objecter ms_dispatch2 0x7faaf0000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 3 client.4422.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 3 client.4422.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 783775441 0 0) 0x7f4f7c162950 con 0x7f4f7c055f60 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7f4f78ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7f4f78ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7f4f78ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7f4f78ff9640 10 monclient(hunting): my global_id is 4443 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f7c162f10 con 0x7f4f7c055f60 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 10 client.4422.objecter checking command 1 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 20 client.4422.objecter _get_session s=0x7faabc0a2e90 osd=0 3 2026-02-25T15:01:42.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 20 client.4422.objecter put_session s=0x7faabc0a2e90 osd=0 3 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 15 client.4422.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 20 client.4422.objecter _get_session s=0x7faabc0a2e90 osd=0 3 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 20 client.4422.objecter get_session s=0x7faabc0a2e90 osd=0 3 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 15 client.4422.objecter _session_command_op_assign 0 1 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 20 client.4422.objecter put_session s=0x7faabc0a2e90 osd=0 4 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 10 client.4422.objecter _send_command 1 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 --> v1:10.20.193.90:6808/3221464204 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faabc0c5140 con 0x7faabc0a3540 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 20 client.4422.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7faad805b060 con 0x7faaf0155b20 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faacf7fe640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.197+0000 7faaef7fe640 10 monclient: get_auth_request con 0x7faabc0a3540 auth_method 0 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.198+0000 7f26be7fc640 10 client.4429.objecter ms_handle_connect 0x7f26a00a3570 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.198+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 251610044 0 0) 0x7f5b90161500 con 0x7f5b900b9f80 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.198+0000 7f5b76ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.198+0000 7f5b76ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.198+0000 7f5b76ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.198+0000 7f5b76ffd640 10 monclient(hunting): my global_id is 4446 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.198+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b9015f760 con 0x7f5b900b9f80 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.198+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 565250925 0 0) 0x7f2a981fb140 con 0x7f2a980585e0 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.198+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 >> v1:10.20.193.94:6789/0 conn(0x7f2a980b9f80 legacy=0x7f2a981fc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.198+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 >> v1:10.20.193.93:6789/0 conn(0x7f2a980ba350 legacy=0x7f2a98158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.198+0000 7f2a8d7fa640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.198+0000 7f2a8d7fa640 1 monclient: found mon.a 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a8d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a8d7fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a981fb420 con 0x7f2a980585e0 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a8d7fa640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a8d7fa640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a8d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a9ec55640 5 monclient: authenticate success, global_id 4425 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a9ec55640 10 monclient: _renew_subs 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a9ec55640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a9ec55640 1 -- 10.20.193.90:0/1700082558 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2a981fb700 con 0x7f2a980585e0 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a9ec55640 10 client.4425.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a9ec55640 10 monclient: _renew_subs 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a9ec55640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7faacf7fe640 10 client.4422.objecter ms_handle_connect 0x7faabc0a3540 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a9ec55640 1 -- 10.20.193.90:0/1700082558 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2a981fbcc0 con 0x7f2a980585e0 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a9ec55640 1 librados: init done 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f73c1ffb640 10 client.?.objecter ms_handle_connect 0x7f73d40585e0 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f73c1ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a9ec55640 10 client.4425.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a9ec55640 15 client.4425.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a9ec55640 20 client.4425.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a9ec55640 10 client.4425.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a9ec55640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.251 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f2a9ec55640 1 -- 10.20.193.90:0/1700082558 --> v1:10.20.193.90:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f2a980b9170 con 0x7f2a980585e0 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f7b7effd640 10 client.?.objecter ms_handle_connect 0x7f7b980585e0 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.199+0000 7f7b7effd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.200+0000 7f7b7effd640 10 client.?.objecter ms_handle_connect 0x7f7b980ba350 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.200+0000 7f7b7effd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3107889483 0 0) 0x7fc010003610 con 0x7fc040161640 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc04013a790 con 0x7fc040161640 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fc02c003310 con 0x7fc040161640 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7fc02affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7fc02affd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7fc02affd640 10 monclient(hunting): dump: 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2865836401 0 0) 0x7f4f7c162c30 con 0x7f4f7c059470 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f4f78ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f4f78ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f4f78ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f4f78ff9640 10 monclient(hunting): my global_id is 4468 2026-02-25T15:01:42.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f7c162950 con 0x7f4f7c059470 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2987902023 0 0) 0x7f47cc1fbcb0 con 0x7f47cc0585e0 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4798003610 con 0x7f47cc0585e0 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f47c0003080 con 0x7f47cc0585e0 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f47ab7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f47ab7fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f47ab7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f829ffff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f82a805d990 0x7f82a8136740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45018/0 (socket says 10.20.193.90:45018) 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f829ffff640 1 -- 10.20.193.90:0/4151938375 learned_addr learned my addr 10.20.193.90:0/4151938375 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 44910643 0 0) 0x7f51281614a0 con 0x7f51280b9f80 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f50fffff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f50fffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f50fffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f50fffff640 10 monclient(hunting): my global_id is 4474 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5128161780 con 0x7f51280b9f80 2026-02-25T15:01:42.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7fc84dca3640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fc8480b5da0 0x7fc848160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45002/0 (socket says 10.20.193.90:45002) 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.201+0000 7fc84dca3640 1 -- 10.20.193.90:0/658736106 learned_addr learned my addr 10.20.193.90:0/658736106 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.202+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f515815e210 con 0x7f51580585e0 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.202+0000 7f5145ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.202+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f51480039b0 con 0x7f51580585e0 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.202+0000 7f5145ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.202+0000 7f51477fe640 4 set_mon_vals no callback set 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.202+0000 7f5145ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.202+0000 7f5145ffb640 10 monclient: dump: 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.202+0000 7f1b62ffd640 10 client.?.objecter ms_handle_connect 0x7f1b780ba350 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.202+0000 7f1b62ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.203+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f5148003bc0 con 0x7f51580585e0 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.203+0000 7f829d7fa640 10 client.?.objecter ms_handle_connect 0x7f82a805d990 2026-02-25T15:01:42.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.203+0000 7f829d7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.203+0000 7f515cefe640 10 monclient: get_auth_request con 0x7f5128079f40 auth_method 0 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.204+0000 7f5145ffb640 10 client.4433.objecter ms_handle_connect 0x7f5128079f40 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.204+0000 7fc8367fc640 10 client.?.objecter ms_handle_connect 0x7fc8480b5da0 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.204+0000 7fc8367fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.204+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1383062658 0 0) 0x7fc04013aa70 con 0x7fc040161a10 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.204+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc010003610 con 0x7fc040161a10 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.204+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fc034003510 con 0x7fc040161a10 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.204+0000 7fc02affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.204+0000 7fc02affd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.204+0000 7fc02affd640 10 monclient(hunting): dump: 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f5148095ad0 con 0x7f51580585e0 2026-02-25T15:01:42.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 10 client.4433.objecter ms_dispatch2 0x7f5158000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 3 client.4433.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 3 client.4433.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 10 client.4433.objecter checking command 1 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 20 client.4433.objecter _get_session s=0x7f51280a2e20 osd=8 3 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 20 client.4433.objecter put_session s=0x7f51280a2e20 osd=8 3 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 15 client.4433.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 20 client.4433.objecter _get_session s=0x7f51280a2e20 osd=8 3 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 20 client.4433.objecter get_session s=0x7f51280a2e20 osd=8 3 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 15 client.4433.objecter _session_command_op_assign 8 1 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 20 client.4433.objecter put_session s=0x7f51280a2e20 osd=8 4 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 10 client.4433.objecter _send_command 1 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 --> v1:10.20.193.90:6801/2320036239 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f51280c50d0 con 0x7f51280a34d0 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 20 client.4433.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7f514805b0a0 con 0x7f51580585e0 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f5145ffb640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3302271696 0 0) 0x7f0bf8134570 con 0x7f0bf815d0c0 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f0bdeffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f0bdeffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f0bdeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f0bdeffd640 10 monclient(hunting): my global_id is 4466 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bf8134290 con 0x7f0bf815d0c0 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.205+0000 7f515df00640 10 monclient: get_auth_request con 0x7f51280a34d0 auth_method 0 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.206+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2694223809 0 0) 0x7f5b60003610 con 0x7f5b900ba350 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.206+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5b90161500 con 0x7f5b900ba350 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.206+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5b8c003560 con 0x7f5b900ba350 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.206+0000 7f5b76ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.206+0000 7f5b76ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.206+0000 7f5b76ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.256 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.206+0000 7f5b76ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.206+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 828403892 0 0) 0x7f51280b7060 con 0x7f51280585e0 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.206+0000 7f50fffff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.206+0000 7f50fffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.206+0000 7f50fffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.206+0000 7f50fffff640 10 monclient(hunting): my global_id is 4452 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.206+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51281614a0 con 0x7f51280585e0 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.206+0000 7f5145ffb640 10 client.4433.objecter ms_handle_connect 0x7f51280a34d0 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.211+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2804771102 0 0) 0x7f1b780b23b0 con 0x7f1b780b9f80 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.211+0000 7f1b62ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.211+0000 7f1b62ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.211+0000 7f1b62ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.211+0000 7f1b62ffd640 10 monclient(hunting): my global_id is 4480 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.211+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b48003610 con 0x7f1b780b9f80 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.213+0000 7f1b62ffd640 10 client.?.objecter ms_handle_connect 0x7f1b780585e0 2026-02-25T15:01:42.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.213+0000 7f1b62ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3038885392 0 0) 0x7f47cc162b60 con 0x7f47cc0b9f80 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 >> v1:10.20.193.94:6789/0 conn(0x7f47cc0ba350 legacy=0x7f47cc15acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 >> v1:10.20.193.93:6789/0 conn(0x7f47cc0585e0 legacy=0x7f47cc1fc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47ab7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47ab7fe640 1 monclient: found mon.a 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47ab7fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc162e40 con 0x7f47cc0b9f80 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47ab7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47ab7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47ab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47d1513640 5 monclient: authenticate success, global_id 4434 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47d1513640 10 monclient: _renew_subs 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47d1513640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47d1513640 1 -- 10.20.193.90:0/2148629278 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f47cc163120 con 0x7f47cc0b9f80 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47d1513640 10 client.4434.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47d1513640 10 monclient: _renew_subs 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47d1513640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47d1513640 1 -- 10.20.193.90:0/2148629278 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f47cc1fcb70 con 0x7f47cc0b9f80 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47d1513640 1 librados: init done 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3699277004 0 0) 0x7f73d40b4ea0 con 0x7f73d40585e0 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f73c1ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f73c1ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f73c1ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f73c1ffb640 10 monclient(hunting): my global_id is 4455 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73a4003610 con 0x7f73d40585e0 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3863970540 0 0) 0x7f73d40b2690 con 0x7f73d40ba350 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f73c1ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f73c1ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f73c1ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f73c1ffb640 10 monclient(hunting): my global_id is 4469 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73d40b4ea0 con 0x7f73d40ba350 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47d1513640 10 client.4434.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47d1513640 15 client.4434.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47d1513640 20 client.4434.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47d1513640 10 client.4434.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47d1513640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.215+0000 7f47d1513640 1 -- 10.20.193.90:0/2148629278 --> v1:10.20.193.90:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f47cc1fbf90 con 0x7f47cc0b9f80 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7f829d7fa640 10 client.?.objecter ms_handle_connect 0x7f82a805a310 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7f829d7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7fc8367fc640 10 client.?.objecter ms_handle_connect 0x7fc848131f20 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7fc8367fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2981669939 0 0) 0x7f73d40b23b0 con 0x7f73d40b9f80 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7f73c1ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7f73c1ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7f73c1ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7f73c1ffb640 10 monclient(hunting): my global_id is 4477 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73d40b2690 con 0x7f73d40b9f80 2026-02-25T15:01:42.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2149652136 0 0) 0x7f7b980afc40 con 0x7f7b980ba350 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7f7b7effd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7f7b7effd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7f7b7effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7f7b7effd640 10 monclient(hunting): my global_id is 4483 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.216+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b64003610 con 0x7f7b980ba350 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.217+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 187434993 0 0) 0x7f7b980b1970 con 0x7f7b980b9f80 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.217+0000 7f7b7effd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.217+0000 7f7b7effd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.217+0000 7f7b7effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.217+0000 7f7b7effd640 10 monclient(hunting): my global_id is 4472 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.217+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b980afc40 con 0x7f7b980b9f80 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.218+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2791538729 0 0) 0x7f4f7c162950 con 0x7f4f7c059470 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.218+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4f7c162c30 con 0x7f4f7c059470 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.218+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4f6c0030b0 con 0x7f4f7c059470 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.218+0000 7f4f78ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.218+0000 7f4f78ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.218+0000 7f4f78ff9640 10 monclient(hunting): dump: 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.218+0000 7f4f78ff9640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.218+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 <== osd.1 v1:10.20.193.93:6804/3161035071 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f26a00c5170 con 0x7f26a00a3570 2026-02-25T15:01:42.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.218+0000 7f26be7fc640 10 client.4429.objecter ms_dispatch2 0x7f26d0000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.218+0000 7f26be7fc640 10 client.4429.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.218+0000 7f26be7fc640 20 client.4429.objecter put_session s=0x7f26a00a2ec0 osd=1 4 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.218+0000 7f26be7fc640 15 client.4429.objecter _session_command_op_remove 1 1 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1921711650 0 0) 0x7f0bf8133fb0 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0bf8134570 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0be8002fb0 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7f0bdeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7f0bdeffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7f0bdeffd640 10 monclient(hunting): dump: 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7f0bdeffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1987375807 0 0) 0x7f5b9015fa40 con 0x7f5b900585e0 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5b60003610 con 0x7f5b900585e0 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5b80002d00 con 0x7f5b900585e0 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7f5b76ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7f5b76ffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7f5b76ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.261 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3228309045 0 0) 0x7fc04013a790 con 0x7fc040161640 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 >> v1:10.20.193.94:6789/0 conn(0x7fc0400b5da0 legacy=0x7fc04013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 >> v1:10.20.193.90:6789/0 conn(0x7fc040161a10 legacy=0x7fc04011e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7fc02affd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.219+0000 7fc02affd640 1 monclient: found mon.b 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc02affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc02affd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04013ad50 con 0x7fc040161640 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc02affd640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc02affd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc02affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc048973640 5 monclient: authenticate success, global_id 4456 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc048973640 10 monclient: _renew_subs 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc048973640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc048973640 1 -- 10.20.193.90:0/3934146879 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc04013b030 con 0x7fc040161640 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc048973640 10 client.4456.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc048973640 10 monclient: _renew_subs 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc048973640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc048973640 1 -- 10.20.193.90:0/3934146879 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc04013bc60 con 0x7fc040161640 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc048973640 1 librados: init done 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc048973640 10 client.4456.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc048973640 15 client.4456.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc048973640 20 client.4456.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc048973640 10 client.4456.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc048973640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.220+0000 7fc048973640 1 -- 10.20.193.90:0/3934146879 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fc0400b2e60 con 0x7fc040161640 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.221+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 613170087 0 0) 0x7f1b780b4ea0 con 0x7f1b780ba350 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.221+0000 7f1b62ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.221+0000 7f1b62ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.221+0000 7f1b62ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.221+0000 7f1b62ffd640 10 monclient(hunting): my global_id is 4461 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.221+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b780b23b0 con 0x7f1b780ba350 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.221+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2299127446 0 0) 0x7f82a81635c0 con 0x7f82a805d990 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.221+0000 7f829d7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.221+0000 7f829d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.221+0000 7f829d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.221+0000 7f829d7fa640 10 monclient(hunting): my global_id is 4464 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.221+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8278003610 con 0x7f82a805d990 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.222+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 <== osd.0 v1:10.20.193.90:6808/3221464204 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7faabc0c5140 con 0x7faabc0a3540 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.222+0000 7faacf7fe640 10 client.4422.objecter ms_dispatch2 0x7faaf0000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.222+0000 7faacf7fe640 10 client.4422.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.222+0000 7faacf7fe640 20 client.4422.objecter put_session s=0x7faabc0a2e90 osd=0 4 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.222+0000 7faacf7fe640 15 client.4422.objecter _session_command_op_remove 0 1 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.222+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fee18060200 con 0x7fee18059e70 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.222+0000 7fee05ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.222+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fee0c003960 con 0x7fee18059e70 2026-02-25T15:01:42.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.222+0000 7fee05ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.222+0000 7fee05ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.222+0000 7fee077fe640 4 set_mon_vals no callback set 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.222+0000 7fee05ffb640 10 monclient: dump: 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.222+0000 7f829d7fa640 10 client.?.objecter ms_handle_connect 0x7f82a8159d10 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.222+0000 7f829d7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.223+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fee0c003b30 con 0x7fee18059e70 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.223+0000 7fee1d23e640 10 monclient: get_auth_request con 0x7fede807a0f0 auth_method 0 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.223+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 489791195 0 0) 0x7f5128161780 con 0x7f51280b9f80 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.223+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f51280b7060 con 0x7f51280b9f80 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.223+0000 7fee05ffb640 10 client.4442.objecter ms_handle_connect 0x7fede807a0f0 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.223+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5110002ff0 con 0x7f51280b9f80 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.223+0000 7f50fffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7f50fffff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7f50fffff640 10 monclient(hunting): dump: 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7f50fffff640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7fee0c095a20 con 0x7fee18059e70 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 10 client.4442.objecter ms_dispatch2 0x7fee18000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 3 client.4442.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 3 client.4442.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 10 client.4442.objecter checking command 1 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 20 client.4442.objecter _get_session s=0x7fede80a2f70 osd=6 3 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 20 client.4442.objecter put_session s=0x7fede80a2f70 osd=6 3 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 15 client.4442.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 20 client.4442.objecter _get_session s=0x7fede80a2f70 osd=6 3 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 20 client.4442.objecter get_session s=0x7fede80a2f70 osd=6 3 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 15 client.4442.objecter _session_command_op_assign 6 1 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 20 client.4442.objecter put_session s=0x7fede80a2f70 osd=6 4 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 10 client.4442.objecter _send_command 1 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 --> v1:10.20.193.94:6812/1114158993 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fee0c002fc0 con 0x7fede80a3620 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 20 client.4442.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7fee0c05aff0 con 0x7fee18059e70 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fee05ffb640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fc8367fc640 10 client.?.objecter ms_handle_connect 0x7fc84815b0a0 2026-02-25T15:01:42.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.224+0000 7fc8367fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7fee1da3f640 10 monclient: get_auth_request con 0x7fede80a3620 auth_method 0 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2438917206 0 0) 0x7f5b90161500 con 0x7f5b900ba350 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 >> v1:10.20.193.93:6789/0 conn(0x7f5b900585e0 legacy=0x7f5b90160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 >> v1:10.20.193.90:6789/0 conn(0x7f5b900b9f80 legacy=0x7f5b901564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b76ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b76ffd640 1 monclient: found mon.c 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b76ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b76ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9015fd20 con 0x7f5b900ba350 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b76ffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b76ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b9869f640 5 monclient: authenticate success, global_id 4463 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b9869f640 10 monclient: _renew_subs 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b9869f640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b9869f640 1 -- 10.20.193.90:0/2131604963 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5b90160000 con 0x7f5b900ba350 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b9869f640 10 client.4463.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b9869f640 10 monclient: _renew_subs 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b9869f640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b9869f640 1 -- 10.20.193.90:0/2131604963 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f5b901605c0 con 0x7f5b900ba350 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b9869f640 1 librados: init done 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3590459926 0 0) 0x7f1b780b2690 con 0x7f1b780585e0 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f1b62ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f1b62ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f1b62ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f1b62ffd640 10 monclient(hunting): my global_id is 4475 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b780b4ea0 con 0x7f1b780585e0 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b9869f640 10 client.4463.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b9869f640 15 client.4463.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b9869f640 20 client.4463.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b9869f640 10 client.4463.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b9869f640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.225+0000 7f5b9869f640 1 -- 10.20.193.90:0/2131604963 --> v1:10.20.193.94:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f5b90056500 con 0x7f5b900ba350 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3074784540 0 0) 0x7f82a80b0210 con 0x7f82a805a310 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f829d7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f829d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f829d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f829d7fa640 10 monclient(hunting): my global_id is 4486 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82a81635c0 con 0x7f82a805a310 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3286613764 0 0) 0x7f50f8003610 con 0x7f51280ba350 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5128161780 con 0x7f51280ba350 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f51180036d0 con 0x7f51280ba350 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f50fffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f50fffff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f50fffff640 10 monclient(hunting): dump: 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.267 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1229763823 0 0) 0x7f4f7c162c30 con 0x7f4f7c059470 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 >> v1:10.20.193.94:6789/0 conn(0x7f4f7c15b220 legacy=0x7f4f7c25d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 >> v1:10.20.193.90:6789/0 conn(0x7f4f7c055f60 legacy=0x7f4f7c150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f78ff9640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f78ff9640 1 monclient: found mon.b 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f78ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f78ff9640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7c1631f0 con 0x7f4f7c059470 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f78ff9640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f78ff9640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f78ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f826b1640 5 monclient: authenticate success, global_id 4468 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f826b1640 10 monclient: _renew_subs 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f826b1640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f826b1640 1 -- 10.20.193.90:0/429045463 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7c1634d0 con 0x7f4f7c059470 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7fee05ffb640 10 client.4442.objecter ms_handle_connect 0x7fede80a3620 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f826b1640 10 client.4468.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f826b1640 10 monclient: _renew_subs 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f826b1640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f826b1640 1 -- 10.20.193.90:0/429045463 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4f7c1fcd50 con 0x7f4f7c059470 2026-02-25T15:01:42.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f4f826b1640 1 librados: init done 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f26d7c5b640 10 client.4429.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f26d7c5b640 15 client.4429.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f26d7c5b640 20 client.4429.objecter _get_session s=0x7f26a00a2ec0 osd=1 3 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.226+0000 7f26d7c5b640 20 client.4429.objecter put_session s=0x7f26a00a2ec0 osd=1 3 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f26d7c5b640 20 client.4429.objecter _get_session s=0x7f26a00a2ec0 osd=1 3 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f26d7c5b640 15 client.4429.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f26d7c5b640 20 client.4429.objecter get_session s=0x7f26a00a2ec0 osd=1 3 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f26d7c5b640 15 client.4429.objecter _session_command_op_assign 1 2 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f26d7c5b640 20 client.4429.objecter put_session s=0x7f26a00a2ec0 osd=1 4 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f26d7c5b640 10 client.4429.objecter _send_command 2 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f26d7c5b640 1 -- 10.20.193.90:0/3380070399 --> v1:10.20.193.93:6804/3161035071 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f26d012abd0 con 0x7f26a00a3570 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f4f826b1640 10 client.4468.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f4f826b1640 15 client.4468.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f4f826b1640 20 client.4468.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f4f826b1640 10 client.4468.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f4f826b1640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f4f826b1640 1 -- 10.20.193.90:0/429045463 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f4f7c1fd030 con 0x7f4f7c059470 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 <== osd.8 v1:10.20.193.90:6801/2320036239 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f51280c50d0 con 0x7f51280a34d0 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f5145ffb640 10 client.4433.objecter ms_dispatch2 0x7f5158000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f5145ffb640 10 client.4433.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f5145ffb640 20 client.4433.objecter put_session s=0x7f51280a2e20 osd=8 4 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f5145ffb640 15 client.4433.objecter _session_command_op_remove 8 1 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2413424599 0 0) 0x7f0bf8134290 con 0x7f0bf815d0c0 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0bf8133fb0 con 0x7f0bf815d0c0 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0bf4003430 con 0x7f0bf815d0c0 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f0bdeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f0bdeffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.227+0000 7f0bdeffd640 10 monclient(hunting): dump: 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1733882873 0 0) 0x7f73d40b2690 con 0x7f73d40b9f80 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f73d40b23b0 con 0x7f73d40b9f80 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f73c8002fb0 con 0x7f73d40b9f80 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7f73c1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7f73c1ffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7f73c1ffb640 10 monclient(hunting): dump: 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7f73c1ffb640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 579643304 0 0) 0x7f7b64003610 con 0x7f7b980ba350 2026-02-25T15:01:42.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7b980b1970 con 0x7f7b980ba350 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7b88002ec0 con 0x7f7b980ba350 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7f7b7effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7f7b7effd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7f7b7effd640 10 monclient(hunting): dump: 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7f7b7effd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7faaf52d1640 10 client.4422.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7faaf52d1640 15 client.4422.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7faaf52d1640 20 client.4422.objecter _get_session s=0x7faabc0a2e90 osd=0 3 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7faaf52d1640 20 client.4422.objecter put_session s=0x7faabc0a2e90 osd=0 3 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7faaf52d1640 20 client.4422.objecter _get_session s=0x7faabc0a2e90 osd=0 3 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7faaf52d1640 15 client.4422.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7faaf52d1640 20 client.4422.objecter get_session s=0x7faabc0a2e90 osd=0 3 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7faaf52d1640 15 client.4422.objecter _session_command_op_assign 0 2 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7faaf52d1640 20 client.4422.objecter put_session s=0x7faabc0a2e90 osd=0 4 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7faaf52d1640 10 client.4422.objecter _send_command 2 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.230+0000 7faaf52d1640 1 -- 10.20.193.90:0/2340633913 --> v1:10.20.193.90:6808/3221464204 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faaf015cc30 con 0x7faabc0a3540 2026-02-25T15:01:42.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 993372340 0 0) 0x7f0bf8134570 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 >> v1:10.20.193.94:6789/0 conn(0x7f0bf815d0c0 legacy=0x7f0bf815ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 >> v1:10.20.193.90:6789/0 conn(0x7f0bf80b28b0 legacy=0x7f0bf81563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0bdeffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0bdeffd640 1 monclient: found mon.b 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0bdeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0bdeffd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf8134850 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0bdeffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0bdeffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0bdeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0c00599640 5 monclient: authenticate success, global_id 4465 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0c00599640 10 monclient: _renew_subs 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0c00599640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0c00599640 1 -- 10.20.193.90:0/1950449469 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf8134b30 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0c00599640 10 client.4465.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0c00599640 10 monclient: _renew_subs 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0c00599640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0c00599640 1 -- 10.20.193.90:0/1950449469 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0bf8155e60 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0c00599640 1 librados: init done 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0c00599640 10 client.4465.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0c00599640 15 client.4465.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0c00599640 20 client.4465.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0c00599640 10 client.4465.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0c00599640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f0c00599640 1 -- 10.20.193.90:0/1950449469 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f0bf805ac10 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f26be7fc640 1 -- 10.20.193.90:0/3380070399 <== osd.1 v1:10.20.193.93:6804/3161035071 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2879543249) 0x7f26d012abd0 con 0x7f26a00a3570 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f26be7fc640 10 client.4429.objecter ms_dispatch2 0x7f26d0000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f26be7fc640 10 client.4429.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f26be7fc640 20 client.4429.objecter put_session s=0x7f26a00a2ec0 osd=1 4 2026-02-25T15:01:42.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.231+0000 7f26be7fc640 15 client.4429.objecter _session_command_op_remove 1 2 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 10 librados: watch_flush enter 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 10 librados: watch_flush exit 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 10 client.4429.objecter close_session for osd.1 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 1 -- 10.20.193.90:0/3380070399 >> v1:10.20.193.93:6804/3161035071 conn(0x7f26a00a3570 legacy=0x7f26a00c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 20 client.4429.objecter put_session s=0x7f26a00a2ec0 osd=1 1 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 20 client.4429.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 10 client.4429.objecter successfully canceled tick 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 1 -- 10.20.193.90:0/3380070399 >> v1:10.20.193.189:6800/3647375982 conn(0x7f26a0079fe0 legacy=0x7f26a009a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 10 monclient: shutdown 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 1 -- 10.20.193.90:0/3380070399 >> v1:10.20.193.93:6789/0 conn(0x7f26d0164d00 legacy=0x7f26d00a9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2a980b9170 con 0x7f2a980585e0 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f2a8d7fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d61d1640 1 -- 10.20.193.90:0/3380070399 reap_dead start 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2a88002fb0 con 0x7f2a980585e0 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f2a8d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f2a8d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f2a8d7fa640 10 monclient: dump: 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.273 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f2a8effd640 4 set_mon_vals no callback set 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 1 -- 10.20.193.90:0/3380070399 shutdown_connections 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 1 -- 10.20.193.90:0/3380070399 >> 10.20.193.90:0/3380070399 conn(0x7f26d00b2ef0 msgr2=0x7f26d00b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 423803188 0 0) 0x7f7b981ff3d0 con 0x7f7b980585e0 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f7b7effd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f7b7effd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f7b7effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f7b7effd640 10 monclient(hunting): my global_id is 4458 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b64003610 con 0x7f7b980585e0 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 1 -- 10.20.193.90:0/3380070399 shutdown_connections 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 1 -- 10.20.193.90:0/3380070399 wait complete. 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f26d7c5b640 1 librados: shutdown 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.232+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f2a88003a50 con 0x7f2a980585e0 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.233+0000 7f2a8ffff640 10 monclient: get_auth_request con 0x7f2a6c07a110 auth_method 0 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.233+0000 7f2a8d7fa640 10 client.4425.objecter ms_handle_connect 0x7f2a6c07a110 2026-02-25T15:01:42.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faacf7fe640 1 -- 10.20.193.90:0/2340633913 <== osd.0 v1:10.20.193.90:6808/3221464204 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2879543249) 0x7faaf015cc30 con 0x7faabc0a3540 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faacf7fe640 10 client.4422.objecter ms_dispatch2 0x7faaf0000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faacf7fe640 10 client.4422.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faacf7fe640 20 client.4422.objecter put_session s=0x7faabc0a2e90 osd=0 4 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faacf7fe640 15 client.4422.objecter _session_command_op_remove 0 2 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 10 librados: watch_flush enter 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 10 librados: watch_flush exit 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 10 client.4422.objecter close_session for osd.0 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 1 -- 10.20.193.90:0/2340633913 >> v1:10.20.193.90:6808/3221464204 conn(0x7faabc0a3540 legacy=0x7faabc0c3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 20 client.4422.objecter put_session s=0x7faabc0a2e90 osd=0 1 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 20 client.4422.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 10 client.4422.objecter successfully canceled tick 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 1 -- 10.20.193.90:0/2340633913 >> v1:10.20.193.189:6800/3647375982 conn(0x7faabc079fb0 legacy=0x7faabc09a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 10 monclient: shutdown 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 1 -- 10.20.193.90:0/2340633913 >> v1:10.20.193.90:6789/0 conn(0x7faaf0155b20 legacy=0x7faaf005ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaef7fe640 1 -- 10.20.193.90:0/2340633913 reap_dead start 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 1 -- 10.20.193.90:0/2340633913 shutdown_connections 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 1 -- 10.20.193.90:0/2340633913 >> 10.20.193.90:0/2340633913 conn(0x7faaf00b2ef0 msgr2=0x7faaf00570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 1 -- 10.20.193.90:0/2340633913 shutdown_connections 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 1 -- 10.20.193.90:0/2340633913 wait complete. 2026-02-25T15:01:42.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.234+0000 7faaf52d1640 1 librados: shutdown 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f515f98a640 10 client.4433.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f515f98a640 15 client.4433.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f515f98a640 20 client.4433.objecter _get_session s=0x7f51280a2e20 osd=8 3 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f515f98a640 20 client.4433.objecter put_session s=0x7f51280a2e20 osd=8 3 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f515f98a640 20 client.4433.objecter _get_session s=0x7f51280a2e20 osd=8 3 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f515f98a640 15 client.4433.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f515f98a640 20 client.4433.objecter get_session s=0x7f51280a2e20 osd=8 3 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f515f98a640 15 client.4433.objecter _session_command_op_assign 8 2 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f515f98a640 20 client.4433.objecter put_session s=0x7f51280a2e20 osd=8 4 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f515f98a640 10 client.4433.objecter _send_command 2 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f515f98a640 1 -- 10.20.193.90:0/2774738662 --> v1:10.20.193.90:6801/2320036239 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5158157dc0 con 0x7f51280a34d0 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2508712993 0 0) 0x7fc848134ef0 con 0x7fc8480b5da0 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7fc8367fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7fc8367fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7fc8367fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7fc8367fc640 10 monclient(hunting): my global_id is 4467 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc818003610 con 0x7fc8480b5da0 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f2a88002fb0 con 0x7f2a980585e0 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 10 client.4425.objecter ms_dispatch2 0x7f2a98000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 3 client.4425.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 3 client.4425.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 10 client.4425.objecter checking command 1 2026-02-25T15:01:42.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 20 client.4425.objecter _get_session s=0x7f2a6c0a2f90 osd=9 3 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 20 client.4425.objecter put_session s=0x7f2a6c0a2f90 osd=9 3 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 15 client.4425.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 20 client.4425.objecter _get_session s=0x7f2a6c0a2f90 osd=9 3 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 20 client.4425.objecter get_session s=0x7f2a6c0a2f90 osd=9 3 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 15 client.4425.objecter _session_command_op_assign 9 1 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 20 client.4425.objecter put_session s=0x7f2a6c0a2f90 osd=9 4 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 10 client.4425.objecter _send_command 1 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 --> v1:10.20.193.93:6808/3039017972 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2a6c0c5240 con 0x7f2a6c0a3640 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 20 client.4425.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7f2a8805e790 con 0x7f2a980585e0 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.235+0000 7f2a8d7fa640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.236+0000 7f2a9d1cb640 10 monclient: get_auth_request con 0x7f2a6c0a3640 auth_method 0 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f2a8d7fa640 10 client.4425.objecter ms_handle_connect 0x7f2a6c0a3640 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f5145ffb640 1 -- 10.20.193.90:0/2774738662 <== osd.8 v1:10.20.193.90:6801/2320036239 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2879543249) 0x7f5158157dc0 con 0x7f51280a34d0 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f5145ffb640 10 client.4433.objecter ms_dispatch2 0x7f5158000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f5145ffb640 10 client.4433.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f5145ffb640 20 client.4433.objecter put_session s=0x7f51280a2e20 osd=8 4 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f5145ffb640 15 client.4433.objecter _session_command_op_remove 8 2 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f515f98a640 10 librados: watch_flush enter 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f515f98a640 10 librados: watch_flush exit 2026-02-25T15:01:42.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f515f98a640 10 client.4433.objecter close_session for osd.8 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f515f98a640 1 -- 10.20.193.90:0/2774738662 >> v1:10.20.193.90:6801/2320036239 conn(0x7f51280a34d0 legacy=0x7f51280c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f515f98a640 20 client.4433.objecter put_session s=0x7f51280a2e20 osd=8 1 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f515f98a640 20 client.4433.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f515f98a640 10 client.4433.objecter successfully canceled tick 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f515f98a640 1 -- 10.20.193.90:0/2774738662 >> v1:10.20.193.189:6800/3647375982 conn(0x7f5128079f40 legacy=0x7f512809a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f515f98a640 10 monclient: shutdown 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f515f98a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f515f98a640 1 -- 10.20.193.90:0/2774738662 >> v1:10.20.193.94:6789/0 conn(0x7f51580585e0 legacy=0x7f5158159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.237+0000 7f515df00640 1 -- 10.20.193.90:0/2774738662 reap_dead start 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.238+0000 7f515f98a640 1 -- 10.20.193.90:0/2774738662 shutdown_connections 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.238+0000 7f515f98a640 1 -- 10.20.193.90:0/2774738662 >> 10.20.193.90:0/2774738662 conn(0x7f51580b7990 msgr2=0x7f515815ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.238+0000 7f515f98a640 1 -- 10.20.193.90:0/2774738662 shutdown_connections 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.238+0000 7f515f98a640 1 -- 10.20.193.90:0/2774738662 wait complete. 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.238+0000 7f515f98a640 1 librados: shutdown 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3970494733 0 0) 0x7f51280b7060 con 0x7f51280b9f80 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 >> v1:10.20.193.94:6789/0 conn(0x7f51280ba350 legacy=0x7f51281559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 >> v1:10.20.193.90:6789/0 conn(0x7f51280585e0 legacy=0x7f51281fc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.278 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f50fffff640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f50fffff640 1 monclient: found mon.b 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f50fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f50fffff640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5128161a60 con 0x7f51280b9f80 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f50fffff640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f50fffff640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f50fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f512d9b3640 5 monclient: authenticate success, global_id 4474 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f512d9b3640 10 monclient: _renew_subs 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f512d9b3640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f512d9b3640 1 -- 10.20.193.90:0/1178662605 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5128161d40 con 0x7f51280b9f80 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3984097766 0 0) 0x7fc848160720 con 0x7fc848131f20 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7fc8367fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7fc8367fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7fc8367fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7fc8367fc640 10 monclient(hunting): my global_id is 4489 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f512d9b3640 10 client.4474.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f512d9b3640 10 monclient: _renew_subs 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f512d9b3640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f512d9b3640 1 -- 10.20.193.90:0/1178662605 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f5128161020 con 0x7f51280b9f80 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f512d9b3640 1 librados: init done 2026-02-25T15:01:42.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc848134ef0 con 0x7fc848131f20 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f47b40041d0 con 0x7f47cc0b9f80 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f47ab7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f47b4002870 con 0x7f47cc0b9f80 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f47ab7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f47ab7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f47ab7fe640 10 monclient: dump: 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f47c97fa640 4 set_mon_vals no callback set 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2957766314 0 0) 0x7fc848133b70 con 0x7fc84815b0a0 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7fc8367fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7fc8367fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7fc8367fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7fc8367fc640 10 monclient(hunting): my global_id is 4481 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc848160720 con 0x7fc84815b0a0 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f512d9b3640 10 client.4474.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f512d9b3640 15 client.4474.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f512d9b3640 20 client.4474.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f512d9b3640 10 client.4474.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f512d9b3640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.240+0000 7f512d9b3640 1 -- 10.20.193.90:0/1178662605 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f5128159660 con 0x7f51280b9f80 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.241+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 616720560 0 0) 0x7f82a81635c0 con 0x7f82a805a310 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.241+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f82a80b0210 con 0x7f82a805a310 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.241+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8298002d00 con 0x7f82a805a310 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.241+0000 7f829d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.241+0000 7f829d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.241+0000 7f829d7fa640 10 monclient(hunting): dump: 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.241+0000 7f829d7fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.241+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f47b4002a80 con 0x7f47cc0b9f80 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.241+0000 7f47caffd640 10 monclient: get_auth_request con 0x7f4798079f50 auth_method 0 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.242+0000 7f47ab7fe640 10 client.4434.objecter ms_handle_connect 0x7f4798079f50 2026-02-25T15:01:42.281 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:42.282 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f47b4095b30 con 0x7f47cc0b9f80 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 10 client.4434.objecter ms_dispatch2 0x7f47cc000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 3 client.4434.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 3 client.4434.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 10 client.4434.objecter checking command 1 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 20 client.4434.objecter _get_session s=0x7f47980a2dd0 osd=11 3 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 20 client.4434.objecter put_session s=0x7f47980a2dd0 osd=11 3 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 15 client.4434.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 20 client.4434.objecter _get_session s=0x7f47980a2dd0 osd=11 3 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 20 client.4434.objecter get_session s=0x7f47980a2dd0 osd=11 3 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 15 client.4434.objecter _session_command_op_assign 11 1 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 20 client.4434.objecter put_session s=0x7f47980a2dd0 osd=11 4 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 10 client.4434.objecter _send_command 1 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 --> v1:10.20.193.189:6801/4031575111 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47980c5120 con 0x7f47980a3480 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 20 client.4434.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7f47b405eb60 con 0x7f47cc0b9f80 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47ab7fe640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f47cb7fe640 10 monclient: get_auth_request con 0x7f47980a3480 auth_method 0 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5b90056500 con 0x7f5b900ba350 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f5b76ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5b8c004c50 con 0x7f5b900ba350 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f5b76ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f5b76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f5b76ffd640 10 monclient: dump: 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.244+0000 7f5b94c11640 4 set_mon_vals no callback set 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.245+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3656210576 0 0) 0x7f1b48003610 con 0x7f1b780b9f80 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.245+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1b780b2690 con 0x7f1b780b9f80 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.245+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1b640030c0 con 0x7f1b780b9f80 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.245+0000 7f1b62ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.245+0000 7f1b62ffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.245+0000 7f1b62ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.245+0000 7f1b62ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.284 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:42.284 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-25T15:01:42.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.245+0000 7f47ab7fe640 10 client.4434.objecter ms_handle_connect 0x7f47980a3480 2026-02-25T15:01:42.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.245+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f5b8c004e60 con 0x7f5b900ba350 2026-02-25T15:01:42.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.246+0000 7f5b96414640 10 monclient: get_auth_request con 0x7f5b60079eb0 auth_method 0 2026-02-25T15:01:42.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.246+0000 7f5b76ffd640 10 client.4463.objecter ms_handle_connect 0x7f5b60079eb0 2026-02-25T15:01:42.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== osd.6 v1:10.20.193.94:6812/1114158993 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7fee0c002fc0 con 0x7fede80a3620 2026-02-25T15:01:42.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7fee05ffb640 10 client.4442.objecter ms_dispatch2 0x7fee18000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7fee05ffb640 10 client.4442.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7fee05ffb640 20 client.4442.objecter put_session s=0x7fede80a2f70 osd=6 4 2026-02-25T15:01:42.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7fee05ffb640 15 client.4442.objecter _session_command_op_remove 6 1 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f5b8c095d00 con 0x7f5b900ba350 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7f5b76ffd640 10 client.4463.objecter ms_dispatch2 0x7f5b90000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7f5b76ffd640 3 client.4463.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7f5b76ffd640 3 client.4463.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7f5b76ffd640 10 client.4463.objecter checking command 1 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7f5b76ffd640 20 client.4463.objecter _get_session s=0x7f5b600a2d30 osd=5 3 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7f5b76ffd640 20 client.4463.objecter put_session s=0x7f5b600a2d30 osd=5 3 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7f5b76ffd640 15 client.4463.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7f5b76ffd640 20 client.4463.objecter _get_session s=0x7f5b600a2d30 osd=5 3 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7f5b76ffd640 20 client.4463.objecter get_session s=0x7f5b600a2d30 osd=5 3 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7f5b76ffd640 15 client.4463.objecter _session_command_op_assign 5 1 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7f5b76ffd640 20 client.4463.objecter put_session s=0x7f5b600a2d30 osd=5 4 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7f5b76ffd640 10 client.4463.objecter _send_command 1 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 --> v1:10.20.193.93:6800/887686167 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b600c5080 con 0x7f5b600a33e0 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.247+0000 7f5b76ffd640 20 client.4463.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f5b95c13640 10 monclient: get_auth_request con 0x7f5b600a33e0 auth_method 0 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3581988485 0 0) 0x7f73d40b23b0 con 0x7f73d40b9f80 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 >> v1:10.20.193.94:6789/0 conn(0x7f73d40ba350 legacy=0x7f73d41fe480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 >> v1:10.20.193.90:6789/0 conn(0x7f73d40585e0 legacy=0x7f73d40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73c1ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73c1ffb640 1 monclient: found mon.b 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73c1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73c1ffb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d40b2970 con 0x7f73d40b9f80 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73c1ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73c1ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73c1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73db755640 5 monclient: authenticate success, global_id 4477 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73db755640 10 monclient: _renew_subs 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73db755640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73db755640 1 -- 10.20.193.90:0/1294973177 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f73d40b2c50 con 0x7f73d40b9f80 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73db755640 10 client.4477.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73db755640 10 monclient: _renew_subs 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73db755640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73db755640 1 -- 10.20.193.90:0/1294973177 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f73d40b3210 con 0x7f73d40b9f80 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.248+0000 7f73db755640 1 librados: init done 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3342048327 0 0) 0x7f7b980b1970 con 0x7f7b980ba350 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 >> v1:10.20.193.94:6789/0 conn(0x7f7b980b9f80 legacy=0x7f7b9815af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 >> v1:10.20.193.90:6789/0 conn(0x7f7b980585e0 legacy=0x7f7b980b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b7effd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b7effd640 1 monclient: found mon.b 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b7effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b7effd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9815b650 con 0x7f7b980ba350 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b7effd640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b7effd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b7effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b9cda2640 5 monclient: authenticate success, global_id 4483 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b9cda2640 10 monclient: _renew_subs 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b9cda2640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b9cda2640 1 -- 10.20.193.90:0/174363358 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7b980afee0 con 0x7f7b980ba350 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b9cda2640 10 client.4483.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b9cda2640 10 monclient: _renew_subs 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b9cda2640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b9cda2640 1 -- 10.20.193.90:0/174363358 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7b980b04a0 con 0x7f7b980ba350 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b9cda2640 1 librados: init done 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f73db755640 10 client.4477.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f73db755640 15 client.4477.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f73db755640 20 client.4477.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f73db755640 10 client.4477.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f73db755640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f73db755640 1 -- 10.20.193.90:0/1294973177 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f73d40b64c0 con 0x7f73d40b9f80 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7f5b8c05b2d0 con 0x7f5b900ba350 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f5b76ffd640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b9cda2640 10 client.4483.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b9cda2640 15 client.4483.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b9cda2640 20 client.4483.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b9cda2640 10 client.4483.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b9cda2640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.249+0000 7f7b9cda2640 1 -- 10.20.193.90:0/174363358 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f7b9804b790 con 0x7f7b980ba350 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.250+0000 7f5b76ffd640 10 client.4463.objecter ms_handle_connect 0x7f5b600a33e0 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.250+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 21402266 0 0) 0x7f1b780b4ea0 con 0x7f1b780585e0 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.250+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1b48003610 con 0x7f1b780585e0 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.250+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1b700034e0 con 0x7f1b780585e0 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.250+0000 7f1b62ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.250+0000 7f1b62ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.250+0000 7f1b62ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.250+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955532041 0 0) 0x7f82a80b04f0 con 0x7f82a8159d10 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.250+0000 7f829d7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.250+0000 7f829d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.250+0000 7f829d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.250+0000 7f829d7fa640 10 monclient(hunting): my global_id is 4478 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.250+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82a81635c0 con 0x7f82a8159d10 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.253+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1049185691 0 0) 0x7f8278003610 con 0x7f82a805d990 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.253+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f82a80b04f0 con 0x7f82a805d990 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.253+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8290003160 con 0x7f82a805d990 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.253+0000 7f829d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.253+0000 7f829d7fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.253+0000 7f829d7fa640 10 monclient(hunting): dump: 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.288 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 839135374 0 0) 0x7f1b48003610 con 0x7f1b780585e0 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 >> v1:10.20.193.93:6789/0 conn(0x7f1b780b9f80 legacy=0x7f1b781fc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 >> v1:10.20.193.90:6789/0 conn(0x7f1b780ba350 legacy=0x7f1b781fe480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b62ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b62ffd640 1 monclient: found mon.c 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b62ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b62ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b780b2970 con 0x7f1b780585e0 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b62ffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b62ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b62ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b808e9640 5 monclient: authenticate success, global_id 4475 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b808e9640 10 monclient: _renew_subs 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b808e9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b808e9640 1 -- 10.20.193.90:0/2348609463 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1b780b2c50 con 0x7f1b780585e0 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b808e9640 10 client.4475.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b808e9640 10 monclient: _renew_subs 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b808e9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b808e9640 1 -- 10.20.193.90:0/2348609463 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f1b780b3210 con 0x7f1b780585e0 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b808e9640 1 librados: init done 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b808e9640 10 client.4475.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b808e9640 15 client.4475.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b808e9640 20 client.4475.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b808e9640 10 client.4475.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b808e9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.255+0000 7f1b808e9640 1 -- 10.20.193.90:0/2348609463 --> v1:10.20.193.94:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f1b780b64c0 con 0x7f1b780585e0 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.256+0000 7fee1f4c9640 10 client.4442.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.256+0000 7fee1f4c9640 15 client.4442.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.256+0000 7fee1f4c9640 20 client.4442.objecter _get_session s=0x7fede80a2f70 osd=6 3 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.256+0000 7fee1f4c9640 20 client.4442.objecter put_session s=0x7fede80a2f70 osd=6 3 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.256+0000 7fee1f4c9640 20 client.4442.objecter _get_session s=0x7fede80a2f70 osd=6 3 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.256+0000 7fee1f4c9640 15 client.4442.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.256+0000 7fee1f4c9640 20 client.4442.objecter get_session s=0x7fede80a2f70 osd=6 3 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.256+0000 7fee1f4c9640 15 client.4442.objecter _session_command_op_assign 6 2 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.256+0000 7fee1f4c9640 20 client.4442.objecter put_session s=0x7fede80a2f70 osd=6 4 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.256+0000 7fee1f4c9640 10 client.4442.objecter _send_command 2 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.256+0000 7fee1f4c9640 1 -- 10.20.193.90:0/1306701788 --> v1:10.20.193.94:6812/1114158993 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fee18136160 con 0x7fede80a3620 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.257+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== osd.9 v1:10.20.193.93:6808/3039017972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f2a6c0c5240 con 0x7f2a6c0a3640 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.257+0000 7f2a8d7fa640 10 client.4425.objecter ms_dispatch2 0x7f2a98000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.257+0000 7f2a8d7fa640 10 client.4425.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.257+0000 7f2a8d7fa640 20 client.4425.objecter put_session s=0x7f2a6c0a2f90 osd=9 4 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.257+0000 7f2a8d7fa640 15 client.4425.objecter _session_command_op_remove 9 1 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee05ffb640 1 -- 10.20.193.90:0/1306701788 <== osd.6 v1:10.20.193.94:6812/1114158993 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2879543249) 0x7fee18136160 con 0x7fede80a3620 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee05ffb640 10 client.4442.objecter ms_dispatch2 0x7fee18000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee05ffb640 10 client.4442.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee05ffb640 20 client.4442.objecter put_session s=0x7fede80a2f70 osd=6 4 2026-02-25T15:01:42.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee05ffb640 15 client.4442.objecter _session_command_op_remove 6 2 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee1f4c9640 10 librados: watch_flush enter 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee1f4c9640 10 librados: watch_flush exit 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee1f4c9640 10 client.4442.objecter close_session for osd.6 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee1f4c9640 1 -- 10.20.193.90:0/1306701788 >> v1:10.20.193.94:6812/1114158993 conn(0x7fede80a3620 legacy=0x7fede80c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee1f4c9640 20 client.4442.objecter put_session s=0x7fede80a2f70 osd=6 1 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee1f4c9640 20 client.4442.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee1f4c9640 10 client.4442.objecter successfully canceled tick 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee1f4c9640 1 -- 10.20.193.90:0/1306701788 >> v1:10.20.193.189:6800/3647375982 conn(0x7fede807a0f0 legacy=0x7fede809a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee1f4c9640 10 monclient: shutdown 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee1f4c9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee1f4c9640 1 -- 10.20.193.90:0/1306701788 >> v1:10.20.193.94:6789/0 conn(0x7fee18059e70 legacy=0x7fee18148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.258+0000 7fee1da3f640 1 -- 10.20.193.90:0/1306701788 reap_dead start 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.259+0000 7fee1f4c9640 1 -- 10.20.193.90:0/1306701788 shutdown_connections 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.259+0000 7fee1f4c9640 1 -- 10.20.193.90:0/1306701788 >> 10.20.193.90:0/1306701788 conn(0x7fee180590f0 msgr2=0x7fee180594c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.259+0000 7fee1f4c9640 1 -- 10.20.193.90:0/1306701788 shutdown_connections 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.259+0000 7fee1f4c9640 1 -- 10.20.193.90:0/1306701788 wait complete. 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.259+0000 7fee1f4c9640 1 librados: shutdown 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.261+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2620319591 0 0) 0x7f82a81635c0 con 0x7f82a8159d10 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.261+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8278003610 con 0x7f82a8159d10 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.261+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f82a4003280 con 0x7f82a8159d10 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.261+0000 7f829d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.261+0000 7f829d7fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.261+0000 7f829d7fa640 10 monclient(hunting): dump: 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.291 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.261+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 679179887 0 0) 0x7fc848160720 con 0x7fc84815b0a0 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.261+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc848133b70 con 0x7fc84815b0a0 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.261+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fc83c003350 con 0x7fc84815b0a0 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.261+0000 7fc8367fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.261+0000 7fc8367fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.261+0000 7fc8367fc640 10 monclient(hunting): dump: 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.261+0000 7fc8367fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.264+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc0400b2e60 con 0x7fc040161640 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.264+0000 7fc02affd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.264+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fc02c0028e0 con 0x7fc040161640 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.264+0000 7fc02affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.264+0000 7fc02affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.264+0000 7fc02affd640 10 monclient: dump: 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.292 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.264+0000 7fc044ee5640 4 set_mon_vals no callback set 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.264+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 <== osd.11 v1:10.20.193.189:6801/4031575111 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f47980c5120 con 0x7f47980a3480 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.264+0000 7f47ab7fe640 10 client.4434.objecter ms_dispatch2 0x7f47cc000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.264+0000 7f47ab7fe640 10 client.4434.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.264+0000 7f47ab7fe640 20 client.4434.objecter put_session s=0x7f47980a2dd0 osd=11 4 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.264+0000 7f47ab7fe640 15 client.4434.objecter _session_command_op_remove 11 1 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2943175140 0 0) 0x7fc818003610 con 0x7fc8480b5da0 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc848160720 con 0x7fc8480b5da0 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fc838003060 con 0x7fc8480b5da0 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7fc8367fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7fc8367fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7fc8367fc640 10 monclient(hunting): dump: 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.293 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2742226584 0 0) 0x7f82a80b04f0 con 0x7f82a805d990 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 >> v1:10.20.193.94:6789/0 conn(0x7f82a8159d10 legacy=0x7f82a80b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 >> v1:10.20.193.93:6789/0 conn(0x7f82a805a310 legacy=0x7f82a811e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f829d7fa640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f829d7fa640 1 monclient: found mon.a 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f829d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f829d7fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a80b07d0 con 0x7f82a805d990 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f829d7fa640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f829d7fa640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f829d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f82aecd0640 5 monclient: authenticate success, global_id 4464 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f82aecd0640 10 monclient: _renew_subs 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f82aecd0640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f82aecd0640 1 -- 10.20.193.90:0/4151938375 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f82a80b0ab0 con 0x7f82a805d990 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f82aecd0640 10 client.4464.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f82aecd0640 10 monclient: _renew_subs 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f82aecd0640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f82aecd0640 1 -- 10.20.193.90:0/4151938375 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f82a8135eb0 con 0x7f82a805d990 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f82aecd0640 1 librados: init done 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fc02c01dec0 con 0x7fc040161640 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f2a9ec55640 10 client.4425.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f2a9ec55640 15 client.4425.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f2a9ec55640 20 client.4425.objecter _get_session s=0x7f2a6c0a2f90 osd=9 3 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f2a9ec55640 20 client.4425.objecter put_session s=0x7f2a6c0a2f90 osd=9 3 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f2a9ec55640 20 client.4425.objecter _get_session s=0x7f2a6c0a2f90 osd=9 3 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f2a9ec55640 15 client.4425.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f2a9ec55640 20 client.4425.objecter get_session s=0x7f2a6c0a2f90 osd=9 3 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f2a9ec55640 15 client.4425.objecter _session_command_op_assign 9 2 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f2a9ec55640 20 client.4425.objecter put_session s=0x7f2a6c0a2f90 osd=9 4 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f2a9ec55640 10 client.4425.objecter _send_command 2 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.265+0000 7f2a9ec55640 1 -- 10.20.193.90:0/1700082558 --> v1:10.20.193.93:6808/3039017972 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2a980b8e90 con 0x7f2a6c0a3640 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.266+0000 7f82aecd0640 10 client.4464.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.266+0000 7f82aecd0640 15 client.4464.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.266+0000 7f82aecd0640 20 client.4464.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.266+0000 7f82aecd0640 10 client.4464.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.266+0000 7f82aecd0640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.266+0000 7f82aecd0640 1 -- 10.20.193.90:0/4151938375 --> v1:10.20.193.90:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f82a80b0e80 con 0x7f82a805d990 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.266+0000 7fc0466e8640 10 monclient: get_auth_request con 0x7fc01007a160 auth_method 0 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.266+0000 7fc02affd640 10 client.4456.objecter ms_handle_connect 0x7fc01007a160 2026-02-25T15:01:42.295 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:42.296 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7fc02c0956c0 con 0x7fc040161640 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc02affd640 10 client.4456.objecter ms_dispatch2 0x7fc040000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc02affd640 3 client.4456.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc02affd640 3 client.4456.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc02affd640 10 client.4456.objecter checking command 1 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc02affd640 20 client.4456.objecter _get_session s=0x7fc0100a2fe0 osd=3 3 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc02affd640 20 client.4456.objecter put_session s=0x7fc0100a2fe0 osd=3 3 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc02affd640 15 client.4456.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc02affd640 20 client.4456.objecter _get_session s=0x7fc0100a2fe0 osd=3 3 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc02affd640 20 client.4456.objecter get_session s=0x7fc0100a2fe0 osd=3 3 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc02affd640 15 client.4456.objecter _session_command_op_assign 3 1 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc02affd640 20 client.4456.objecter put_session s=0x7fc0100a2fe0 osd=3 4 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc02affd640 10 client.4456.objecter _send_command 1 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 --> v1:10.20.193.189:6813/2702305154 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc0100c5290 con 0x7fc0100a3690 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc02affd640 20 client.4456.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.267+0000 7fc046ee9640 10 monclient: get_auth_request con 0x7fc0100a3690 auth_method 0 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1867592614 0 0) 0x7fc848160720 con 0x7fc8480b5da0 2026-02-25T15:01:42.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a8d7fa640 1 -- 10.20.193.90:0/1700082558 <== osd.9 v1:10.20.193.93:6808/3039017972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2879543249) 0x7f2a980b8e90 con 0x7f2a6c0a3640 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 >> v1:10.20.193.94:6789/0 conn(0x7fc84815b0a0 legacy=0x7fc848141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 >> v1:10.20.193.93:6789/0 conn(0x7fc848131f20 legacy=0x7fc848132330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a8d7fa640 10 client.4425.objecter ms_dispatch2 0x7f2a98000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a8d7fa640 10 client.4425.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a8d7fa640 20 client.4425.objecter put_session s=0x7f2a6c0a2f90 osd=9 4 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a8d7fa640 15 client.4425.objecter _session_command_op_remove 9 2 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc8367fc640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc8367fc640 1 monclient: found mon.a 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc8367fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc8367fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc848133e50 con 0x7fc8480b5da0 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc8367fc640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc8367fc640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc8367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc84ff2e640 5 monclient: authenticate success, global_id 4467 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc84ff2e640 10 monclient: _renew_subs 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc84ff2e640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc84ff2e640 1 -- 10.20.193.90:0/658736106 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc848134130 con 0x7fc8480b5da0 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc84ff2e640 10 client.4467.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc84ff2e640 10 monclient: _renew_subs 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc84ff2e640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc84ff2e640 1 -- 10.20.193.90:0/658736106 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc8481346f0 con 0x7fc8480b5da0 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7fc84ff2e640 1 librados: init done 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 10 librados: watch_flush enter 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 10 librados: watch_flush exit 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 10 client.4425.objecter close_session for osd.9 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 1 -- 10.20.193.90:0/1700082558 >> v1:10.20.193.93:6808/3039017972 conn(0x7f2a6c0a3640 legacy=0x7f2a6c0c3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 20 client.4425.objecter put_session s=0x7f2a6c0a2f90 osd=9 1 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 20 client.4425.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 10 client.4425.objecter successfully canceled tick 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 1 -- 10.20.193.90:0/1700082558 >> v1:10.20.193.189:6800/3647375982 conn(0x7f2a6c07a110 legacy=0x7f2a6c09a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 10 monclient: shutdown 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 1 -- 10.20.193.90:0/1700082558 >> v1:10.20.193.90:6789/0 conn(0x7f2a980585e0 legacy=0x7f2a980b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9d1cb640 1 -- 10.20.193.90:0/1700082558 reap_dead start 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 1 -- 10.20.193.90:0/1700082558 shutdown_connections 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 1 -- 10.20.193.90:0/1700082558 >> 10.20.193.90:0/1700082558 conn(0x7f2a980b7990 msgr2=0x7f2a9815bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 1 -- 10.20.193.90:0/1700082558 shutdown_connections 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 1 -- 10.20.193.90:0/1700082558 wait complete. 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.268+0000 7f2a9ec55640 1 librados: shutdown 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7fc84ff2e640 10 client.4467.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7fc84ff2e640 15 client.4467.objecter _session_command_op_assign -1 1 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7fc84ff2e640 20 client.4467.objecter _get_session osd=-1 returning homeless 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7fc84ff2e640 10 client.4467.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7fc84ff2e640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7fc84ff2e640 1 -- 10.20.193.90:0/658736106 --> v1:10.20.193.90:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fc848055260 con 0x7fc8480b5da0 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7fc02affd640 10 client.4456.objecter ms_handle_connect 0x7fc0100a3690 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7fc02c05e6f0 con 0x7fc040161640 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7fc02affd640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1b780b64c0 con 0x7f1b780585e0 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7f1b62ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1b70003b30 con 0x7f1b780585e0 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7f1b62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7f1b62ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7f1b62ffd640 10 monclient: dump: 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.269+0000 7f1b7ce5b640 4 set_mon_vals no callback set 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.270+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f1b7001d010 con 0x7f1b780585e0 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.270+0000 7f1b7de5d640 10 monclient: get_auth_request con 0x7f1b48079fe0 auth_method 0 2026-02-25T15:01:42.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.271+0000 7f1b62ffd640 10 client.4475.objecter ms_handle_connect 0x7f1b48079fe0 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.271+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 <== osd.5 v1:10.20.193.93:6800/887686167 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f5b600c5080 con 0x7f5b600a33e0 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.271+0000 7f5b76ffd640 10 client.4463.objecter ms_dispatch2 0x7f5b90000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.271+0000 7f5b76ffd640 10 client.4463.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.271+0000 7f5b76ffd640 20 client.4463.objecter put_session s=0x7f5b600a2d30 osd=5 4 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.271+0000 7f5b76ffd640 15 client.4463.objecter _session_command_op_remove 5 1 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f1b700948d0 con 0x7f1b780585e0 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b62ffd640 10 client.4475.objecter ms_dispatch2 0x7f1b78000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b62ffd640 3 client.4475.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b62ffd640 3 client.4475.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b62ffd640 10 client.4475.objecter checking command 1 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b62ffd640 20 client.4475.objecter _get_session s=0x7f1b480a2e60 osd=12 3 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b62ffd640 20 client.4475.objecter put_session s=0x7f1b480a2e60 osd=12 3 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b62ffd640 15 client.4475.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b62ffd640 20 client.4475.objecter _get_session s=0x7f1b480a2e60 osd=12 3 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b62ffd640 20 client.4475.objecter get_session s=0x7f1b480a2e60 osd=12 3 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b62ffd640 15 client.4475.objecter _session_command_op_assign 12 1 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b62ffd640 20 client.4475.objecter put_session s=0x7f1b480a2e60 osd=12 4 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b62ffd640 10 client.4475.objecter _send_command 1 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 --> v1:10.20.193.90:6812/1139796254 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b480c5110 con 0x7f1b480a3510 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b62ffd640 20 client.4475.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f1b7ee5f640 10 monclient: get_auth_request con 0x7f1b480a3510 auth_method 0 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.272+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f82a80b0e80 con 0x7f82a805d990 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.273+0000 7f829d7fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.273+0000 7f829effd640 4 set_mon_vals no callback set 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.273+0000 7f47d1513640 10 client.4434.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.273+0000 7f47d1513640 15 client.4434.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.273+0000 7f47d1513640 20 client.4434.objecter _get_session s=0x7f47980a2dd0 osd=11 3 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.273+0000 7f47d1513640 20 client.4434.objecter put_session s=0x7f47980a2dd0 osd=11 3 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.273+0000 7f47d1513640 20 client.4434.objecter _get_session s=0x7f47980a2dd0 osd=11 3 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.273+0000 7f47d1513640 15 client.4434.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.273+0000 7f47d1513640 20 client.4434.objecter get_session s=0x7f47980a2dd0 osd=11 3 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.273+0000 7f47d1513640 15 client.4434.objecter _session_command_op_assign 11 2 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.273+0000 7f47d1513640 20 client.4434.objecter put_session s=0x7f47980a2dd0 osd=11 4 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.273+0000 7f47d1513640 10 client.4434.objecter _send_command 2 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.273+0000 7f47d1513640 1 -- 10.20.193.90:0/2148629278 --> v1:10.20.193.189:6801/4031575111 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f47cc0b64c0 con 0x7f47980a3480 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.273+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7f1b70059ea0 con 0x7f1b780585e0 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.274+0000 7f1b62ffd640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.274+0000 7f1b62ffd640 10 client.4475.objecter ms_handle_connect 0x7f1b480a3510 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.274+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4f6c003700 con 0x7f4f7c059470 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.274+0000 7f4f78ff9640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.274+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4f6c003a50 con 0x7f4f7c059470 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.274+0000 7f4f78ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.274+0000 7f4f78ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.274+0000 7f4f78ff9640 10 monclient: dump: 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.301 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.302 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.302 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.302 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.302 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.302 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.274+0000 7f4f7a7fc640 4 set_mon_vals no callback set 2026-02-25T15:01:42.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.275+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f4f6c01cf30 con 0x7f4f7c059470 2026-02-25T15:01:42.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.275+0000 7f4f7bfff640 10 monclient: get_auth_request con 0x7f4f4c079f60 auth_method 0 2026-02-25T15:01:42.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.275+0000 7f4f78ff9640 10 client.4468.objecter ms_handle_connect 0x7f4f4c079f60 2026-02-25T15:01:42.302 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:42.302 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-25T15:01:42.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.276+0000 7f47ab7fe640 1 -- 10.20.193.90:0/2148629278 <== osd.11 v1:10.20.193.189:6801/4031575111 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2879543249) 0x7f47cc0b64c0 con 0x7f47980a3480 2026-02-25T15:01:42.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.276+0000 7f47ab7fe640 10 client.4434.objecter ms_dispatch2 0x7f47cc000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.276+0000 7f47ab7fe640 10 client.4434.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47ab7fe640 20 client.4434.objecter put_session s=0x7f47980a2dd0 osd=11 4 2026-02-25T15:01:42.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47ab7fe640 15 client.4434.objecter _session_command_op_remove 11 2 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 10 librados: watch_flush enter 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 10 librados: watch_flush exit 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 10 client.4434.objecter close_session for osd.11 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 1 -- 10.20.193.90:0/2148629278 >> v1:10.20.193.189:6801/4031575111 conn(0x7f47980a3480 legacy=0x7f47980c38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 20 client.4434.objecter put_session s=0x7f47980a2dd0 osd=11 1 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 20 client.4434.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 10 client.4434.objecter successfully canceled tick 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 1 -- 10.20.193.90:0/2148629278 >> v1:10.20.193.189:6800/3647375982 conn(0x7f4798079f50 legacy=0x7f479809a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 10 monclient: shutdown 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 1 -- 10.20.193.90:0/2148629278 >> v1:10.20.193.90:6789/0 conn(0x7f47cc0b9f80 legacy=0x7f47cc1fb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47cb7fe640 1 -- 10.20.193.90:0/2148629278 reap_dead start 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 1 -- 10.20.193.90:0/2148629278 shutdown_connections 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 1 -- 10.20.193.90:0/2148629278 >> 10.20.193.90:0/2148629278 conn(0x7f47cc0b7990 msgr2=0x7f47cc158420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 1 -- 10.20.193.90:0/2148629278 shutdown_connections 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 1 -- 10.20.193.90:0/2148629278 wait complete. 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f47d1513640 1 librados: shutdown 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f82900039b0 con 0x7f82a805d990 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f829d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f829d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f829d7fa640 10 monclient: dump: 2026-02-25T15:01:42.303 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f4f6c094a40 con 0x7f4f7c059470 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f4f78ff9640 10 client.4468.objecter ms_dispatch2 0x7f4f7c000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f4f78ff9640 3 client.4468.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f4f78ff9640 3 client.4468.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f4f78ff9640 10 client.4468.objecter checking command 1 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f4f78ff9640 20 client.4468.objecter _get_session s=0x7f4f4c0a2de0 osd=2 3 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f4f78ff9640 20 client.4468.objecter put_session s=0x7f4f4c0a2de0 osd=2 3 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f4f78ff9640 15 client.4468.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f4f78ff9640 20 client.4468.objecter _get_session s=0x7f4f4c0a2de0 osd=2 3 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f4f78ff9640 20 client.4468.objecter get_session s=0x7f4f4c0a2de0 osd=2 3 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f4f78ff9640 15 client.4468.objecter _session_command_op_assign 2 1 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f4f78ff9640 20 client.4468.objecter put_session s=0x7f4f4c0a2de0 osd=2 4 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f4f78ff9640 10 client.4468.objecter _send_command 1 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 --> v1:10.20.193.94:6800/3705139403 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f4c0c5090 con 0x7f4f4c0a3490 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.277+0000 7f4f78ff9640 20 client.4468.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.278+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7f4f6c05a010 con 0x7f4f7c059470 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.278+0000 7f4f78ff9640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.278+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f82900039b0 con 0x7f82a805d990 2026-02-25T15:01:42.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.278+0000 7f4f80c27640 10 monclient: get_auth_request con 0x7f4f4c0a3490 auth_method 0 2026-02-25T15:01:42.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.278+0000 7f82aca45640 10 monclient: get_auth_request con 0x7f827807a0e0 auth_method 0 2026-02-25T15:01:42.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.279+0000 7f829d7fa640 10 client.4464.objecter ms_handle_connect 0x7f827807a0e0 2026-02-25T15:01:42.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.279+0000 7f5b9869f640 10 client.4463.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.279+0000 7f5b9869f640 15 client.4463.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.279+0000 7f5b9869f640 20 client.4463.objecter _get_session s=0x7f5b600a2d30 osd=5 3 2026-02-25T15:01:42.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.279+0000 7f5b9869f640 20 client.4463.objecter put_session s=0x7f5b600a2d30 osd=5 3 2026-02-25T15:01:42.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.279+0000 7f5b9869f640 20 client.4463.objecter _get_session s=0x7f5b600a2d30 osd=5 3 2026-02-25T15:01:42.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.279+0000 7f5b9869f640 15 client.4463.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.279+0000 7f5b9869f640 20 client.4463.objecter get_session s=0x7f5b600a2d30 osd=5 3 2026-02-25T15:01:42.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.279+0000 7f5b9869f640 15 client.4463.objecter _session_command_op_assign 5 2 2026-02-25T15:01:42.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.279+0000 7f5b9869f640 20 client.4463.objecter put_session s=0x7f5b600a2d30 osd=5 4 2026-02-25T15:01:42.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.279+0000 7f5b9869f640 10 client.4463.objecter _send_command 2 2026-02-25T15:01:42.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.279+0000 7f5b9869f640 1 -- 10.20.193.90:0/2131604963 --> v1:10.20.193.93:6800/887686167 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5b900b64c0 con 0x7f5b600a33e0 2026-02-25T15:01:42.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.279+0000 7f4f78ff9640 10 client.4468.objecter ms_handle_connect 0x7f4f4c0a3490 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.280+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f829005ab90 con 0x7f82a805d990 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.280+0000 7f829d7fa640 10 client.4464.objecter ms_dispatch2 0x7f82a8000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.280+0000 7f829d7fa640 3 client.4464.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.280+0000 7f829d7fa640 3 client.4464.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.281+0000 7f829d7fa640 10 client.4464.objecter checking command 1 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.281+0000 7f829d7fa640 20 client.4464.objecter _get_session s=0x7f82780a2f60 osd=14 3 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.281+0000 7f829d7fa640 20 client.4464.objecter put_session s=0x7f82780a2f60 osd=14 3 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.281+0000 7f829d7fa640 15 client.4464.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.281+0000 7f829d7fa640 20 client.4464.objecter _get_session s=0x7f82780a2f60 osd=14 3 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.281+0000 7f829d7fa640 20 client.4464.objecter get_session s=0x7f82780a2f60 osd=14 3 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.281+0000 7f829d7fa640 15 client.4464.objecter _session_command_op_assign 14 1 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.281+0000 7f829d7fa640 20 client.4464.objecter put_session s=0x7f82780a2f60 osd=14 4 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.281+0000 7f829d7fa640 10 client.4464.objecter _send_command 1 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.281+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 --> v1:10.20.193.94:6808/887271598 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f82780c5210 con 0x7f82780a3610 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.281+0000 7f829d7fa640 20 client.4464.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.281+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7f829005af70 con 0x7f82a805d990 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.281+0000 7f829d7fa640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.281+0000 7f82ad246640 10 monclient: get_auth_request con 0x7f82780a3610 auth_method 0 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b76ffd640 1 -- 10.20.193.90:0/2131604963 <== osd.5 v1:10.20.193.93:6800/887686167 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2879543249) 0x7f5b900b64c0 con 0x7f5b600a33e0 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b76ffd640 10 client.4463.objecter ms_dispatch2 0x7f5b90000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b76ffd640 10 client.4463.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b76ffd640 20 client.4463.objecter put_session s=0x7f5b600a2d30 osd=5 4 2026-02-25T15:01:42.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b76ffd640 15 client.4463.objecter _session_command_op_remove 5 2 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 10 librados: watch_flush enter 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 10 librados: watch_flush exit 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 10 client.4463.objecter close_session for osd.5 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 1 -- 10.20.193.90:0/2131604963 >> v1:10.20.193.93:6800/887686167 conn(0x7f5b600a33e0 legacy=0x7f5b600c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 20 client.4463.objecter put_session s=0x7f5b600a2d30 osd=5 1 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 20 client.4463.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 10 client.4463.objecter successfully canceled tick 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 1 -- 10.20.193.90:0/2131604963 >> v1:10.20.193.189:6800/3647375982 conn(0x7f5b60079eb0 legacy=0x7f5b6009a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 10 monclient: shutdown 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 1 -- 10.20.193.90:0/2131604963 >> v1:10.20.193.94:6789/0 conn(0x7f5b900ba350 legacy=0x7f5b90162ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b96c15640 1 -- 10.20.193.90:0/2131604963 reap_dead start 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 1 -- 10.20.193.90:0/2131604963 shutdown_connections 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 1 -- 10.20.193.90:0/2131604963 >> 10.20.193.90:0/2131604963 conn(0x7f5b900b7990 msgr2=0x7f5b90056830 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 1 -- 10.20.193.90:0/2131604963 shutdown_connections 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 1 -- 10.20.193.90:0/2131604963 wait complete. 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f5b9869f640 1 librados: shutdown 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.282+0000 7f829d7fa640 10 client.4464.objecter ms_handle_connect 0x7f82780a3610 2026-02-25T15:01:42.307 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:42.308 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.288+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5128159660 con 0x7f51280b9f80 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.288+0000 7f50fffff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.288+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5110003700 con 0x7f51280b9f80 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.288+0000 7f50fffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.288+0000 7f50fffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.288+0000 7f50fffff640 10 monclient: dump: 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.288+0000 7f51257fa640 4 set_mon_vals no callback set 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.289+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== osd.3 v1:10.20.193.189:6813/2702305154 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7fc0100c5290 con 0x7fc0100a3690 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.289+0000 7fc02affd640 10 client.4456.objecter ms_dispatch2 0x7fc040000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.289+0000 7fc02affd640 10 client.4456.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.289+0000 7fc02affd640 20 client.4456.objecter put_session s=0x7fc0100a2fe0 osd=3 4 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.289+0000 7fc02affd640 15 client.4456.objecter _session_command_op_remove 3 1 2026-02-25T15:01:42.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.289+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f5110003910 con 0x7f51280b9f80 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.290+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc848055260 con 0x7fc8480b5da0 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.290+0000 7fc8367fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.290+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fc8380038a0 con 0x7fc8480b5da0 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.290+0000 7fc8367fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.290+0000 7fc8367fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.309 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.290+0000 7fc8367fc640 10 monclient: dump: 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.290+0000 7fc837fff640 4 set_mon_vals no callback set 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.290+0000 7f5126ffd640 10 monclient: get_auth_request con 0x7f50f807a0f0 auth_method 0 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.290+0000 7f50fffff640 10 client.4474.objecter ms_handle_connect 0x7f50f807a0f0 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.290+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fc838003ab0 con 0x7fc8480b5da0 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.291+0000 7fc84d4a2640 10 monclient: get_auth_request con 0x7fc818079f90 auth_method 0 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.291+0000 7fc8367fc640 10 client.4467.objecter ms_handle_connect 0x7fc818079f90 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7fc838095a00 con 0x7fc8480b5da0 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 10 client.4467.objecter ms_dispatch2 0x7fc848000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 3 client.4467.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 3 client.4467.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 10 client.4467.objecter checking command 1 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 20 client.4467.objecter _get_session s=0x7fc8180a2e10 osd=10 3 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 20 client.4467.objecter put_session s=0x7fc8180a2e10 osd=10 3 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 15 client.4467.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 20 client.4467.objecter _get_session s=0x7fc8180a2e10 osd=10 3 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 20 client.4467.objecter get_session s=0x7fc8180a2e10 osd=10 3 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 15 client.4467.objecter _session_command_op_assign 10 1 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 20 client.4467.objecter put_session s=0x7fc8180a2e10 osd=10 4 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 10 client.4467.objecter _send_command 1 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 --> v1:10.20.193.94:6804/4185080112 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc8180c50c0 con 0x7fc8180a34c0 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 20 client.4467.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.292+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7fc83805ea30 con 0x7fc8480b5da0 2026-02-25T15:01:42.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7fc8367fc640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f51100947a0 con 0x7f51280b9f80 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f50fffff640 10 client.4474.objecter ms_dispatch2 0x7f5128000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f50fffff640 3 client.4474.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f50fffff640 3 client.4474.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f50fffff640 10 client.4474.objecter checking command 1 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f50fffff640 20 client.4474.objecter _get_session s=0x7f50f80a2f70 osd=7 3 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f50fffff640 20 client.4474.objecter put_session s=0x7f50f80a2f70 osd=7 3 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f50fffff640 15 client.4474.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f50fffff640 20 client.4474.objecter _get_session s=0x7f50f80a2f70 osd=7 3 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f50fffff640 20 client.4474.objecter get_session s=0x7f50f80a2f70 osd=7 3 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f50fffff640 15 client.4474.objecter _session_command_op_assign 7 1 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f50fffff640 20 client.4474.objecter put_session s=0x7f50f80a2f70 osd=7 4 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f50fffff640 10 client.4474.objecter _send_command 1 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 --> v1:10.20.193.189:6809/1708410075 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5110003190 con 0x7f50f80a3620 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f50fffff640 20 client.4474.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7fc84e4a4640 10 monclient: get_auth_request con 0x7fc8180a34c0 auth_method 0 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.293+0000 7f51277fe640 10 monclient: get_auth_request con 0x7f50f80a3620 auth_method 0 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.294+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 <== osd.12 v1:10.20.193.90:6812/1139796254 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f1b480c5110 con 0x7f1b480a3510 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.294+0000 7f1b62ffd640 10 client.4475.objecter ms_dispatch2 0x7f1b78000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.294+0000 7f1b62ffd640 10 client.4475.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.294+0000 7f1b62ffd640 20 client.4475.objecter put_session s=0x7f1b480a2e60 osd=12 4 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.294+0000 7f1b62ffd640 15 client.4475.objecter _session_command_op_remove 12 1 2026-02-25T15:01:42.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.295+0000 7fc8367fc640 10 client.4467.objecter ms_handle_connect 0x7fc8180a34c0 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.295+0000 7f50fffff640 10 client.4474.objecter ms_handle_connect 0x7f50f80a3620 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7fc048973640 10 client.4456.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7fc048973640 15 client.4456.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7fc048973640 20 client.4456.objecter _get_session s=0x7fc0100a2fe0 osd=3 3 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7fc048973640 20 client.4456.objecter put_session s=0x7fc0100a2fe0 osd=3 3 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7fc048973640 20 client.4456.objecter _get_session s=0x7fc0100a2fe0 osd=3 3 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7fc048973640 15 client.4456.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7fc048973640 20 client.4456.objecter get_session s=0x7fc0100a2fe0 osd=3 3 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7fc048973640 15 client.4456.objecter _session_command_op_assign 3 2 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7fc048973640 20 client.4456.objecter put_session s=0x7fc0100a2fe0 osd=3 4 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7fc048973640 10 client.4456.objecter _send_command 2 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7fc048973640 1 -- 10.20.193.90:0/3934146879 --> v1:10.20.193.189:6813/2702305154 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc04005bbc0 con 0x7fc0100a3690 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0be8003530 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7f0bdeffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0be8004a70 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7f0bdeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7f0bdeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7f0bdeffd640 10 monclient: dump: 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.312 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.297+0000 7f0bfcb0b640 4 set_mon_vals no callback set 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.298+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7f5110059d70 con 0x7f51280b9f80 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.298+0000 7f50fffff640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.298+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f0be801df50 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.299+0000 7f0bfdb0d640 10 monclient: get_auth_request con 0x7f0bc8079f50 auth_method 0 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.299+0000 7f0bdeffd640 10 client.4465.objecter ms_handle_connect 0x7f0bc8079f50 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f0be8095ba0 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 10 client.4465.objecter ms_dispatch2 0x7f0bf8000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 3 client.4465.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 3 client.4465.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 10 client.4465.objecter checking command 1 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 20 client.4465.objecter _get_session s=0x7f0bc80a2db0 osd=15 3 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 20 client.4465.objecter put_session s=0x7f0bc80a2db0 osd=15 3 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 15 client.4465.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 20 client.4465.objecter _get_session s=0x7f0bc80a2db0 osd=15 3 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 20 client.4465.objecter get_session s=0x7f0bc80a2db0 osd=15 3 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 15 client.4465.objecter _session_command_op_assign 15 1 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 20 client.4465.objecter put_session s=0x7f0bc80a2db0 osd=15 4 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 10 client.4465.objecter _send_command 1 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 --> v1:10.20.193.189:6805/4039858964 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0bc80c5060 con 0x7f0bc80a3460 2026-02-25T15:01:42.313 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 20 client.4465.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7fc02affd640 1 -- 10.20.193.90:0/3934146879 <== osd.3 v1:10.20.193.189:6813/2702305154 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2115957317) 0x7fc04005bbc0 con 0x7fc0100a3690 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7fc02affd640 10 client.4456.objecter ms_dispatch2 0x7fc040000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7fc02affd640 10 client.4456.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7fc02affd640 20 client.4456.objecter put_session s=0x7fc0100a2fe0 osd=3 4 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7fc02affd640 15 client.4456.objecter _session_command_op_remove 3 2 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7f0be805ebd0 con 0x7f0bf80b0ca0 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7f0bdeffd640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7fc048973640 10 librados: watch_flush enter 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.300+0000 7fc048973640 10 librados: watch_flush exit 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7fc048973640 10 client.4456.objecter close_session for osd.3 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7fc048973640 1 -- 10.20.193.90:0/3934146879 >> v1:10.20.193.189:6813/2702305154 conn(0x7fc0100a3690 legacy=0x7fc0100c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7fc048973640 20 client.4456.objecter put_session s=0x7fc0100a2fe0 osd=3 1 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7fc048973640 20 client.4456.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7fc048973640 10 client.4456.objecter successfully canceled tick 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7fc048973640 1 -- 10.20.193.90:0/3934146879 >> v1:10.20.193.189:6800/3647375982 conn(0x7fc01007a160 legacy=0x7fc01009a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7fc048973640 10 monclient: shutdown 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7fc048973640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7fc048973640 1 -- 10.20.193.90:0/3934146879 >> v1:10.20.193.93:6789/0 conn(0x7fc040161640 legacy=0x7fc04012ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.314 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7fc046ee9640 1 -- 10.20.193.90:0/3934146879 reap_dead start 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7fc048973640 1 -- 10.20.193.90:0/3934146879 shutdown_connections 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7fc048973640 1 -- 10.20.193.90:0/3934146879 >> 10.20.193.90:0/3934146879 conn(0x7fc0400b5290 msgr2=0x7fc04015ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7fc048973640 1 -- 10.20.193.90:0/3934146879 shutdown_connections 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7fc048973640 1 -- 10.20.193.90:0/3934146879 wait complete. 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7f0bfeb0f640 10 monclient: get_auth_request con 0x7f0bc80a3460 auth_method 0 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7fc048973640 1 librados: shutdown 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 <== osd.2 v1:10.20.193.94:6800/3705139403 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f4f4c0c5090 con 0x7f4f4c0a3490 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7f4f78ff9640 10 client.4468.objecter ms_dispatch2 0x7f4f7c000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7f4f78ff9640 10 client.4468.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7f4f78ff9640 20 client.4468.objecter put_session s=0x7f4f4c0a2de0 osd=2 4 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.301+0000 7f4f78ff9640 15 client.4468.objecter _session_command_op_remove 2 1 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.302+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== osd.14 v1:10.20.193.94:6808/887271598 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f82780c5210 con 0x7f82780a3610 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.302+0000 7f829d7fa640 10 client.4464.objecter ms_dispatch2 0x7f82a8000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.302+0000 7f829d7fa640 10 client.4464.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.302+0000 7f829d7fa640 20 client.4464.objecter put_session s=0x7f82780a2f60 osd=14 4 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.302+0000 7f829d7fa640 15 client.4464.objecter _session_command_op_remove 14 1 2026-02-25T15:01:42.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.302+0000 7f0bdeffd640 10 client.4465.objecter ms_handle_connect 0x7f0bc80a3460 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.303+0000 7f1b808e9640 10 client.4475.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.303+0000 7f1b808e9640 15 client.4475.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.303+0000 7f1b808e9640 20 client.4475.objecter _get_session s=0x7f1b480a2e60 osd=12 3 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.303+0000 7f1b808e9640 20 client.4475.objecter put_session s=0x7f1b480a2e60 osd=12 3 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.303+0000 7f1b808e9640 20 client.4475.objecter _get_session s=0x7f1b480a2e60 osd=12 3 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.303+0000 7f1b808e9640 15 client.4475.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.303+0000 7f1b808e9640 20 client.4475.objecter get_session s=0x7f1b480a2e60 osd=12 3 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.303+0000 7f1b808e9640 15 client.4475.objecter _session_command_op_assign 12 2 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.303+0000 7f1b808e9640 20 client.4475.objecter put_session s=0x7f1b480a2e60 osd=12 4 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.303+0000 7f1b808e9640 10 client.4475.objecter _send_command 2 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.303+0000 7f1b808e9640 1 -- 10.20.193.90:0/2348609463 --> v1:10.20.193.90:6812/1139796254 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b780b94f0 con 0x7f1b480a3510 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.305+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f73d40b64c0 con 0x7f73d40b9f80 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.305+0000 7f73c1ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.305+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f73c8003700 con 0x7f73d40b9f80 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.305+0000 7f73c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.305+0000 7f73c1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.305+0000 7f73c1ffb640 10 monclient: dump: 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.316 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.305+0000 7f73c37fe640 4 set_mon_vals no callback set 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.305+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f73c801cc20 con 0x7f73d40b9f80 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f73d94ca640 10 monclient: get_auth_request con 0x7f73a4079fb0 auth_method 0 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f73c1ffb640 10 client.4477.objecter ms_handle_connect 0x7f73a4079fb0 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b62ffd640 1 -- 10.20.193.90:0/2348609463 <== osd.12 v1:10.20.193.90:6812/1139796254 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 1506399442) 0x7f1b780b94f0 con 0x7f1b480a3510 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b62ffd640 10 client.4475.objecter ms_dispatch2 0x7f1b78000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b62ffd640 10 client.4475.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b62ffd640 20 client.4475.objecter put_session s=0x7f1b480a2e60 osd=12 4 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b62ffd640 15 client.4475.objecter _session_command_op_remove 12 2 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 10 librados: watch_flush enter 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 10 librados: watch_flush exit 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 10 client.4475.objecter close_session for osd.12 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 1 -- 10.20.193.90:0/2348609463 >> v1:10.20.193.90:6812/1139796254 conn(0x7f1b480a3510 legacy=0x7f1b480c3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 20 client.4475.objecter put_session s=0x7f1b480a2e60 osd=12 1 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 20 client.4475.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 10 client.4475.objecter successfully canceled tick 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 1 -- 10.20.193.90:0/2348609463 >> v1:10.20.193.189:6800/3647375982 conn(0x7f1b48079fe0 legacy=0x7f1b4809a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 10 monclient: shutdown 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 1 -- 10.20.193.90:0/2348609463 >> v1:10.20.193.94:6789/0 conn(0x7f1b780585e0 legacy=0x7f1b780b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b7ee5f640 1 -- 10.20.193.90:0/2348609463 reap_dead start 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 1 -- 10.20.193.90:0/2348609463 shutdown_connections 2026-02-25T15:01:42.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 1 -- 10.20.193.90:0/2348609463 >> 10.20.193.90:0/2348609463 conn(0x7f1b780b7990 msgr2=0x7f1b780b66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 1 -- 10.20.193.90:0/2348609463 shutdown_connections 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 1 -- 10.20.193.90:0/2348609463 wait complete. 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.306+0000 7f1b808e9640 1 librados: shutdown 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f73c80947e0 con 0x7f73d40b9f80 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 10 client.4477.objecter ms_dispatch2 0x7f73d4000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 3 client.4477.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 3 client.4477.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 10 client.4477.objecter checking command 1 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 20 client.4477.objecter _get_session s=0x7f73a40a2e10 osd=4 3 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 20 client.4477.objecter put_session s=0x7f73a40a2e10 osd=4 3 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 15 client.4477.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 20 client.4477.objecter _get_session s=0x7f73a40a2e10 osd=4 3 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 20 client.4477.objecter get_session s=0x7f73a40a2e10 osd=4 3 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 15 client.4477.objecter _session_command_op_assign 4 1 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 20 client.4477.objecter put_session s=0x7f73a40a2e10 osd=4 4 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 10 client.4477.objecter _send_command 1 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 --> v1:10.20.193.90:6800/1446526134 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73a40c50c0 con 0x7f73a40a34c0 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 20 client.4477.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7f73c805d810 con 0x7f73d40b9f80 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73c1ffb640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.307+0000 7f73d9ccb640 10 monclient: get_auth_request con 0x7f73a40a34c0 auth_method 0 2026-02-25T15:01:42.319 INFO:teuthology.orchestra.run.trial090.stdout:94489280516 2026-02-25T15:01:42.319 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-25T15:01:42.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.309+0000 7f73c1ffb640 10 client.4477.objecter ms_handle_connect 0x7f73a40a34c0 2026-02-25T15:01:42.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.309+0000 7f4f826b1640 10 client.4468.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.309+0000 7f4f826b1640 15 client.4468.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.309+0000 7f4f826b1640 20 client.4468.objecter _get_session s=0x7f4f4c0a2de0 osd=2 3 2026-02-25T15:01:42.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.309+0000 7f4f826b1640 20 client.4468.objecter put_session s=0x7f4f4c0a2de0 osd=2 3 2026-02-25T15:01:42.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.309+0000 7f4f826b1640 20 client.4468.objecter _get_session s=0x7f4f4c0a2de0 osd=2 3 2026-02-25T15:01:42.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.309+0000 7f4f826b1640 15 client.4468.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.309+0000 7f4f826b1640 20 client.4468.objecter get_session s=0x7f4f4c0a2de0 osd=2 3 2026-02-25T15:01:42.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.309+0000 7f4f826b1640 15 client.4468.objecter _session_command_op_assign 2 2 2026-02-25T15:01:42.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.309+0000 7f4f826b1640 20 client.4468.objecter put_session s=0x7f4f4c0a2de0 osd=2 4 2026-02-25T15:01:42.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.309+0000 7f4f826b1640 10 client.4468.objecter _send_command 2 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.309+0000 7f4f826b1640 1 -- 10.20.193.90:0/429045463 --> v1:10.20.193.94:6800/3705139403 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f7c055260 con 0x7f4f4c0a3490 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f82aecd0640 10 client.4464.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f82aecd0640 15 client.4464.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f82aecd0640 20 client.4464.objecter _get_session s=0x7f82780a2f60 osd=14 3 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f82aecd0640 20 client.4464.objecter put_session s=0x7f82780a2f60 osd=14 3 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f82aecd0640 20 client.4464.objecter _get_session s=0x7f82780a2f60 osd=14 3 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f82aecd0640 15 client.4464.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f82aecd0640 20 client.4464.objecter get_session s=0x7f82780a2f60 osd=14 3 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f82aecd0640 15 client.4464.objecter _session_command_op_assign 14 2 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f82aecd0640 20 client.4464.objecter put_session s=0x7f82780a2f60 osd=14 4 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f82aecd0640 10 client.4464.objecter _send_command 2 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f82aecd0640 1 -- 10.20.193.90:0/4151938375 --> v1:10.20.193.94:6808/887271598 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f82a8060200 con 0x7f82780a3610 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7b9804b790 con 0x7f7b980ba350 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f7b7effd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7b880036e0 con 0x7f7b980ba350 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f7b7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f7b7effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f7b7effd640 10 monclient: dump: 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f7b94d72640 4 set_mon_vals no callback set 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.311+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f7b8801cbc0 con 0x7f7b980ba350 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.312+0000 7f7b96575640 10 monclient: get_auth_request con 0x7f7b6407a040 auth_method 0 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.312+0000 7f7b7effd640 10 client.4483.objecter ms_handle_connect 0x7f7b6407a040 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f78ff9640 1 -- 10.20.193.90:0/429045463 <== osd.2 v1:10.20.193.94:6800/3705139403 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2879543249) 0x7f4f7c055260 con 0x7f4f4c0a3490 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f78ff9640 10 client.4468.objecter ms_dispatch2 0x7f4f7c000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f78ff9640 10 client.4468.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f78ff9640 20 client.4468.objecter put_session s=0x7f4f4c0a2de0 osd=2 4 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f78ff9640 15 client.4468.objecter _session_command_op_remove 2 2 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 10 librados: watch_flush enter 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 10 librados: watch_flush exit 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 10 client.4468.objecter close_session for osd.2 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 1 -- 10.20.193.90:0/429045463 >> v1:10.20.193.94:6800/3705139403 conn(0x7f4f4c0a3490 legacy=0x7f4f4c0c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 20 client.4468.objecter put_session s=0x7f4f4c0a2de0 osd=2 1 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 20 client.4468.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 10 client.4468.objecter successfully canceled tick 2026-02-25T15:01:42.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 1 -- 10.20.193.90:0/429045463 >> v1:10.20.193.189:6800/3647375982 conn(0x7f4f4c079f60 legacy=0x7f4f4c09a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 10 monclient: shutdown 2026-02-25T15:01:42.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 1 -- 10.20.193.90:0/429045463 >> v1:10.20.193.93:6789/0 conn(0x7f4f7c059470 legacy=0x7f4f7c1fc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f7b88094760 con 0x7f7b980ba350 2026-02-25T15:01:42.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 10 client.4483.objecter ms_dispatch2 0x7f7b98000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f80c27640 1 -- 10.20.193.90:0/429045463 reap_dead start 2026-02-25T15:01:42.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 3 client.4483.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 3 client.4483.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 1 -- 10.20.193.90:0/429045463 shutdown_connections 2026-02-25T15:01:42.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 1 -- 10.20.193.90:0/429045463 >> 10.20.193.90:0/429045463 conn(0x7f4f7c0b4100 msgr2=0x7f4f7c0b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 1 -- 10.20.193.90:0/429045463 shutdown_connections 2026-02-25T15:01:42.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 10 client.4483.objecter checking command 1 2026-02-25T15:01:42.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 1 -- 10.20.193.90:0/429045463 wait complete. 2026-02-25T15:01:42.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 20 client.4483.objecter _get_session s=0x7f7b640a2ec0 osd=13 3 2026-02-25T15:01:42.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 20 client.4483.objecter put_session s=0x7f7b640a2ec0 osd=13 3 2026-02-25T15:01:42.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 15 client.4483.objecter _session_command_op_remove -1 1 2026-02-25T15:01:42.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 20 client.4483.objecter _get_session s=0x7f7b640a2ec0 osd=13 3 2026-02-25T15:01:42.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 20 client.4483.objecter get_session s=0x7f7b640a2ec0 osd=13 3 2026-02-25T15:01:42.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 15 client.4483.objecter _session_command_op_assign 13 1 2026-02-25T15:01:42.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 20 client.4483.objecter put_session s=0x7f7b640a2ec0 osd=13 4 2026-02-25T15:01:42.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 10 client.4483.objecter _send_command 1 2026-02-25T15:01:42.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 --> v1:10.20.193.93:6812/1850919728 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b640c5170 con 0x7f7b640a3570 2026-02-25T15:01:42.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 20 client.4483.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (unknown 2863928538 0 0) 0x7f7b88059d30 con 0x7f7b980ba350 2026-02-25T15:01:42.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f7b7effd640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-25T15:01:42.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.313+0000 7f4f826b1640 1 librados: shutdown 2026-02-25T15:01:42.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.314+0000 7f7b95d74640 10 monclient: get_auth_request con 0x7f7b640a3570 auth_method 0 2026-02-25T15:01:42.323 INFO:teuthology.orchestra.run.trial090.stdout:90194313221 2026-02-25T15:01:42.323 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.315+0000 7f7b7effd640 10 client.4483.objecter ms_handle_connect 0x7f7b640a3570 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f829d7fa640 1 -- 10.20.193.90:0/4151938375 <== osd.14 v1:10.20.193.94:6808/887271598 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2879543249) 0x7f82a8060200 con 0x7f82780a3610 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f829d7fa640 10 client.4464.objecter ms_dispatch2 0x7f82a8000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f829d7fa640 10 client.4464.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f829d7fa640 20 client.4464.objecter put_session s=0x7f82780a2f60 osd=14 4 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f829d7fa640 15 client.4464.objecter _session_command_op_remove 14 2 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 10 librados: watch_flush enter 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 10 librados: watch_flush exit 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 10 client.4464.objecter close_session for osd.14 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 1 -- 10.20.193.90:0/4151938375 >> v1:10.20.193.94:6808/887271598 conn(0x7f82780a3610 legacy=0x7f82780c3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 20 client.4464.objecter put_session s=0x7f82780a2f60 osd=14 1 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 20 client.4464.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 10 client.4464.objecter successfully canceled tick 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 1 -- 10.20.193.90:0/4151938375 >> v1:10.20.193.189:6800/3647375982 conn(0x7f827807a0e0 legacy=0x7f827809a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 10 monclient: shutdown 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 1 -- 10.20.193.90:0/4151938375 >> v1:10.20.193.90:6789/0 conn(0x7f82a805d990 legacy=0x7f82a8136740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 <== osd.10 v1:10.20.193.94:6804/4185080112 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7fc8180c50c0 con 0x7fc8180a34c0 2026-02-25T15:01:42.325 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7fc8367fc640 10 client.4467.objecter ms_dispatch2 0x7fc848000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.325 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82ad246640 1 -- 10.20.193.90:0/4151938375 reap_dead start 2026-02-25T15:01:42.325 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7fc8367fc640 10 client.4467.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.325 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7fc8367fc640 20 client.4467.objecter put_session s=0x7fc8180a2e10 osd=10 4 2026-02-25T15:01:42.325 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7fc8367fc640 15 client.4467.objecter _session_command_op_remove 10 1 2026-02-25T15:01:42.325 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 1 -- 10.20.193.90:0/4151938375 shutdown_connections 2026-02-25T15:01:42.325 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 1 -- 10.20.193.90:0/4151938375 >> 10.20.193.90:0/4151938375 conn(0x7f82a80590f0 msgr2=0x7f82a80594c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.325 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 1 -- 10.20.193.90:0/4151938375 shutdown_connections 2026-02-25T15:01:42.325 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 1 -- 10.20.193.90:0/4151938375 wait complete. 2026-02-25T15:01:42.325 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.316+0000 7f82aecd0640 1 librados: shutdown 2026-02-25T15:01:42.325 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.317+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 <== osd.7 v1:10.20.193.189:6809/1708410075 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f5110003190 con 0x7f50f80a3620 2026-02-25T15:01:42.325 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.317+0000 7f50fffff640 10 client.4474.objecter ms_dispatch2 0x7f5128000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.325 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.317+0000 7f50fffff640 10 client.4474.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.325 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.317+0000 7f50fffff640 20 client.4474.objecter put_session s=0x7f50f80a2f70 osd=7 4 2026-02-25T15:01:42.325 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.317+0000 7f50fffff640 15 client.4474.objecter _session_command_op_remove 7 1 2026-02-25T15:01:42.326 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:42.326 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-25T15:01:42.326 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.323+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 <== osd.15 v1:10.20.193.189:6805/4039858964 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f0bc80c5060 con 0x7f0bc80a3460 2026-02-25T15:01:42.326 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.323+0000 7f0bdeffd640 10 client.4465.objecter ms_dispatch2 0x7f0bf8000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.326 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.323+0000 7f0bdeffd640 10 client.4465.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.326 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.323+0000 7f0bdeffd640 20 client.4465.objecter put_session s=0x7f0bc80a2db0 osd=15 4 2026-02-25T15:01:42.326 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.323+0000 7f0bdeffd640 15 client.4465.objecter _session_command_op_remove 15 1 2026-02-25T15:01:42.327 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:42.327 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2026-02-25T15:01:42.327 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.325+0000 7f512d9b3640 10 client.4474.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.327 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.325+0000 7f512d9b3640 15 client.4474.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.327 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.325+0000 7f512d9b3640 20 client.4474.objecter _get_session s=0x7f50f80a2f70 osd=7 3 2026-02-25T15:01:42.327 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.325+0000 7f512d9b3640 20 client.4474.objecter put_session s=0x7f50f80a2f70 osd=7 3 2026-02-25T15:01:42.327 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.325+0000 7f512d9b3640 20 client.4474.objecter _get_session s=0x7f50f80a2f70 osd=7 3 2026-02-25T15:01:42.327 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.325+0000 7f512d9b3640 15 client.4474.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.327 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.325+0000 7f512d9b3640 20 client.4474.objecter get_session s=0x7f50f80a2f70 osd=7 3 2026-02-25T15:01:42.327 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.325+0000 7f512d9b3640 15 client.4474.objecter _session_command_op_assign 7 2 2026-02-25T15:01:42.327 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.325+0000 7f512d9b3640 20 client.4474.objecter put_session s=0x7f50f80a2f70 osd=7 4 2026-02-25T15:01:42.327 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.325+0000 7f512d9b3640 10 client.4474.objecter _send_command 2 2026-02-25T15:01:42.327 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.325+0000 7f512d9b3640 1 -- 10.20.193.90:0/1178662605 --> v1:10.20.193.189:6809/1708410075 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f51281673b0 con 0x7f50f80a3620 2026-02-25T15:01:42.328 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.326+0000 7fc84ff2e640 10 client.4467.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.328 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.326+0000 7fc84ff2e640 15 client.4467.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.328 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.326+0000 7fc84ff2e640 20 client.4467.objecter _get_session s=0x7fc8180a2e10 osd=10 3 2026-02-25T15:01:42.328 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.326+0000 7fc84ff2e640 20 client.4467.objecter put_session s=0x7fc8180a2e10 osd=10 3 2026-02-25T15:01:42.328 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.326+0000 7fc84ff2e640 20 client.4467.objecter _get_session s=0x7fc8180a2e10 osd=10 3 2026-02-25T15:01:42.328 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.326+0000 7fc84ff2e640 15 client.4467.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.328 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.326+0000 7fc84ff2e640 20 client.4467.objecter get_session s=0x7fc8180a2e10 osd=10 3 2026-02-25T15:01:42.328 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.326+0000 7fc84ff2e640 15 client.4467.objecter _session_command_op_assign 10 2 2026-02-25T15:01:42.328 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.326+0000 7fc84ff2e640 20 client.4467.objecter put_session s=0x7fc8180a2e10 osd=10 4 2026-02-25T15:01:42.328 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.326+0000 7fc84ff2e640 10 client.4467.objecter _send_command 2 2026-02-25T15:01:42.328 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.326+0000 7fc84ff2e640 1 -- 10.20.193.90:0/658736106 --> v1:10.20.193.94:6804/4185080112 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc8481615a0 con 0x7fc8180a34c0 2026-02-25T15:01:42.329 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f50fffff640 1 -- 10.20.193.90:0/1178662605 <== osd.7 v1:10.20.193.189:6809/1708410075 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2356506950) 0x7f51281673b0 con 0x7f50f80a3620 2026-02-25T15:01:42.329 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f50fffff640 10 client.4474.objecter ms_dispatch2 0x7f5128000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.329 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f50fffff640 10 client.4474.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.329 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f50fffff640 20 client.4474.objecter put_session s=0x7f50f80a2f70 osd=7 4 2026-02-25T15:01:42.329 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f50fffff640 15 client.4474.objecter _session_command_op_remove 7 2 2026-02-25T15:01:42.331 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f512d9b3640 10 librados: watch_flush enter 2026-02-25T15:01:42.331 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f512d9b3640 10 librados: watch_flush exit 2026-02-25T15:01:42.331 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f512d9b3640 10 client.4474.objecter close_session for osd.7 2026-02-25T15:01:42.331 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f512d9b3640 1 -- 10.20.193.90:0/1178662605 >> v1:10.20.193.189:6809/1708410075 conn(0x7f50f80a3620 legacy=0x7f50f80c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.331 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f512d9b3640 20 client.4474.objecter put_session s=0x7f50f80a2f70 osd=7 1 2026-02-25T15:01:42.331 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f512d9b3640 20 client.4474.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.331 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f512d9b3640 10 client.4474.objecter successfully canceled tick 2026-02-25T15:01:42.331 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f512d9b3640 1 -- 10.20.193.90:0/1178662605 >> v1:10.20.193.189:6800/3647375982 conn(0x7f50f807a0f0 legacy=0x7f50f809a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f512d9b3640 10 monclient: shutdown 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f512d9b3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f512d9b3640 1 -- 10.20.193.90:0/1178662605 >> v1:10.20.193.93:6789/0 conn(0x7f51280b9f80 legacy=0x7f51280b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f51277fe640 1 -- 10.20.193.90:0/1178662605 reap_dead start 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f512d9b3640 1 -- 10.20.193.90:0/1178662605 shutdown_connections 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f512d9b3640 1 -- 10.20.193.90:0/1178662605 >> 10.20.193.90:0/1178662605 conn(0x7f51280b7990 msgr2=0x7f5128157c50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7f512d9b3640 1 -- 10.20.193.90:0/1178662605 shutdown_connections 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7f512d9b3640 1 -- 10.20.193.90:0/1178662605 wait complete. 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7f512d9b3640 1 librados: shutdown 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7fc8367fc640 1 -- 10.20.193.90:0/658736106 <== osd.10 v1:10.20.193.94:6804/4185080112 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2879543249) 0x7fc8481615a0 con 0x7fc8180a34c0 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7fc8367fc640 10 client.4467.objecter ms_dispatch2 0x7fc848000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7fc8367fc640 10 client.4467.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7fc8367fc640 20 client.4467.objecter put_session s=0x7fc8180a2e10 osd=10 4 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.328+0000 7fc8367fc640 15 client.4467.objecter _session_command_op_remove 10 2 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 10 librados: watch_flush enter 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 10 librados: watch_flush exit 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 10 client.4467.objecter close_session for osd.10 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 1 -- 10.20.193.90:0/658736106 >> v1:10.20.193.94:6804/4185080112 conn(0x7fc8180a34c0 legacy=0x7fc8180c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 20 client.4467.objecter put_session s=0x7fc8180a2e10 osd=10 1 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 20 client.4467.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 10 client.4467.objecter successfully canceled tick 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 1 -- 10.20.193.90:0/658736106 >> v1:10.20.193.189:6800/3647375982 conn(0x7fc818079f90 legacy=0x7fc81809a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 10 monclient: shutdown 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 1 -- 10.20.193.90:0/658736106 >> v1:10.20.193.90:6789/0 conn(0x7fc8480b5da0 legacy=0x7fc848160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84e4a4640 1 -- 10.20.193.90:0/658736106 reap_dead start 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 1 -- 10.20.193.90:0/658736106 shutdown_connections 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 1 -- 10.20.193.90:0/658736106 >> 10.20.193.90:0/658736106 conn(0x7fc8480b5290 msgr2=0x7fc8481592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 1 -- 10.20.193.90:0/658736106 shutdown_connections 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 1 -- 10.20.193.90:0/658736106 wait complete. 2026-02-25T15:01:42.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.329+0000 7fc84ff2e640 1 librados: shutdown 2026-02-25T15:01:42.333 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.332+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 <== osd.4 v1:10.20.193.90:6800/1446526134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f73a40c50c0 con 0x7f73a40a34c0 2026-02-25T15:01:42.333 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.332+0000 7f73c1ffb640 10 client.4477.objecter ms_dispatch2 0x7f73d4000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.333 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.332+0000 7f73c1ffb640 10 client.4477.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.333 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.332+0000 7f73c1ffb640 20 client.4477.objecter put_session s=0x7f73a40a2e10 osd=4 4 2026-02-25T15:01:42.333 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.332+0000 7f73c1ffb640 15 client.4477.objecter _session_command_op_remove 4 1 2026-02-25T15:01:42.335 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.334+0000 7f0c00599640 10 client.4465.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.335 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.334+0000 7f0c00599640 15 client.4465.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.335 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.334+0000 7f0c00599640 20 client.4465.objecter _get_session s=0x7f0bc80a2db0 osd=15 3 2026-02-25T15:01:42.335 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.334+0000 7f0c00599640 20 client.4465.objecter put_session s=0x7f0bc80a2db0 osd=15 3 2026-02-25T15:01:42.335 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.334+0000 7f0c00599640 20 client.4465.objecter _get_session s=0x7f0bc80a2db0 osd=15 3 2026-02-25T15:01:42.335 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.334+0000 7f0c00599640 15 client.4465.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.335 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.334+0000 7f0c00599640 20 client.4465.objecter get_session s=0x7f0bc80a2db0 osd=15 3 2026-02-25T15:01:42.335 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.334+0000 7f0c00599640 15 client.4465.objecter _session_command_op_assign 15 2 2026-02-25T15:01:42.335 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.334+0000 7f0c00599640 20 client.4465.objecter put_session s=0x7f0bc80a2db0 osd=15 4 2026-02-25T15:01:42.335 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.334+0000 7f0c00599640 10 client.4465.objecter _send_command 2 2026-02-25T15:01:42.335 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.334+0000 7f0c00599640 1 -- 10.20.193.90:0/1950449469 --> v1:10.20.193.189:6805/4039858964 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0bf805e210 con 0x7f0bc80a3460 2026-02-25T15:01:42.337 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.336+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 <== osd.13 v1:10.20.193.93:6812/1850919728 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f7b640c5170 con 0x7f7b640a3570 2026-02-25T15:01:42.337 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.336+0000 7f7b7effd640 10 client.4483.objecter ms_dispatch2 0x7f7b98000b90 command_reply(tid 1: 0 ) 2026-02-25T15:01:42.337 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.336+0000 7f7b7effd640 10 client.4483.objecter _finish_command 1 = system:0 2026-02-25T15:01:42.337 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.336+0000 7f7b7effd640 20 client.4483.objecter put_session s=0x7f7b640a2ec0 osd=13 4 2026-02-25T15:01:42.337 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.336+0000 7f7b7effd640 15 client.4483.objecter _session_command_op_remove 13 1 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.336+0000 7f0bdeffd640 1 -- 10.20.193.90:0/1950449469 <== osd.15 v1:10.20.193.189:6805/4039858964 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2879543249) 0x7f0bf805e210 con 0x7f0bc80a3460 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.336+0000 7f0bdeffd640 10 client.4465.objecter ms_dispatch2 0x7f0bf8000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.336+0000 7f0bdeffd640 10 client.4465.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.336+0000 7f0bdeffd640 20 client.4465.objecter put_session s=0x7f0bc80a2db0 osd=15 4 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.336+0000 7f0bdeffd640 15 client.4465.objecter _session_command_op_remove 15 2 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 10 librados: watch_flush enter 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 10 librados: watch_flush exit 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 10 client.4465.objecter close_session for osd.15 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 1 -- 10.20.193.90:0/1950449469 >> v1:10.20.193.189:6805/4039858964 conn(0x7f0bc80a3460 legacy=0x7f0bc80c3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 20 client.4465.objecter put_session s=0x7f0bc80a2db0 osd=15 1 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 20 client.4465.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 10 client.4465.objecter successfully canceled tick 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 1 -- 10.20.193.90:0/1950449469 >> v1:10.20.193.189:6800/3647375982 conn(0x7f0bc8079f50 legacy=0x7f0bc809a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 10 monclient: shutdown 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.338 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 1 -- 10.20.193.90:0/1950449469 >> v1:10.20.193.93:6789/0 conn(0x7f0bf80b0ca0 legacy=0x7f0bf812b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.339 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0bfeb0f640 1 -- 10.20.193.90:0/1950449469 reap_dead start 2026-02-25T15:01:42.339 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 1 -- 10.20.193.90:0/1950449469 shutdown_connections 2026-02-25T15:01:42.339 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 1 -- 10.20.193.90:0/1950449469 >> 10.20.193.90:0/1950449469 conn(0x7f0bf80b5290 msgr2=0x7f0bf81357d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.339 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 1 -- 10.20.193.90:0/1950449469 shutdown_connections 2026-02-25T15:01:42.339 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 1 -- 10.20.193.90:0/1950449469 wait complete. 2026-02-25T15:01:42.339 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.337+0000 7f0c00599640 1 librados: shutdown 2026-02-25T15:01:42.339 INFO:teuthology.orchestra.run.trial090.stdout:94489280517 2026-02-25T15:01:42.339 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-25T15:01:42.340 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:42.340 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2026-02-25T15:01:42.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.341+0000 7f73db755640 10 client.4477.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.341+0000 7f73db755640 15 client.4477.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.341+0000 7f73db755640 20 client.4477.objecter _get_session s=0x7f73a40a2e10 osd=4 3 2026-02-25T15:01:42.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.341+0000 7f73db755640 20 client.4477.objecter put_session s=0x7f73a40a2e10 osd=4 3 2026-02-25T15:01:42.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.341+0000 7f73db755640 20 client.4477.objecter _get_session s=0x7f73a40a2e10 osd=4 3 2026-02-25T15:01:42.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.341+0000 7f73db755640 15 client.4477.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.341+0000 7f73db755640 20 client.4477.objecter get_session s=0x7f73a40a2e10 osd=4 3 2026-02-25T15:01:42.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.341+0000 7f73db755640 15 client.4477.objecter _session_command_op_assign 4 2 2026-02-25T15:01:42.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.341+0000 7f73db755640 20 client.4477.objecter put_session s=0x7f73a40a2e10 osd=4 4 2026-02-25T15:01:42.343 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.341+0000 7f73db755640 10 client.4477.objecter _send_command 2 2026-02-25T15:01:42.343 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.341+0000 7f73db755640 1 -- 10.20.193.90:0/1294973177 --> v1:10.20.193.90:6800/1446526134 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f73d40b94f0 con 0x7f73a40a34c0 2026-02-25T15:01:42.346 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.345+0000 7f73c1ffb640 1 -- 10.20.193.90:0/1294973177 <== osd.4 v1:10.20.193.90:6800/1446526134 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2879543249) 0x7f73d40b94f0 con 0x7f73a40a34c0 2026-02-25T15:01:42.346 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.345+0000 7f73c1ffb640 10 client.4477.objecter ms_dispatch2 0x7f73d4000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.346 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.345+0000 7f73c1ffb640 10 client.4477.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.346 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.345+0000 7f73c1ffb640 20 client.4477.objecter put_session s=0x7f73a40a2e10 osd=4 4 2026-02-25T15:01:42.346 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.345+0000 7f73c1ffb640 15 client.4477.objecter _session_command_op_remove 4 2 2026-02-25T15:01:42.348 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f7b9cda2640 10 client.4483.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-25T15:01:42.348 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f7b9cda2640 15 client.4483.objecter _session_command_op_assign -1 2 2026-02-25T15:01:42.348 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f7b9cda2640 20 client.4483.objecter _get_session s=0x7f7b640a2ec0 osd=13 3 2026-02-25T15:01:42.348 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f7b9cda2640 20 client.4483.objecter put_session s=0x7f7b640a2ec0 osd=13 3 2026-02-25T15:01:42.348 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f7b9cda2640 20 client.4483.objecter _get_session s=0x7f7b640a2ec0 osd=13 3 2026-02-25T15:01:42.348 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f7b9cda2640 15 client.4483.objecter _session_command_op_remove -1 2 2026-02-25T15:01:42.348 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f7b9cda2640 20 client.4483.objecter get_session s=0x7f7b640a2ec0 osd=13 3 2026-02-25T15:01:42.348 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f7b9cda2640 15 client.4483.objecter _session_command_op_assign 13 2 2026-02-25T15:01:42.348 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f7b9cda2640 20 client.4483.objecter put_session s=0x7f7b640a2ec0 osd=13 4 2026-02-25T15:01:42.348 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f7b9cda2640 10 client.4483.objecter _send_command 2 2026-02-25T15:01:42.348 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f7b9cda2640 1 -- 10.20.193.90:0/174363358 --> v1:10.20.193.93:6812/1850919728 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7b981593c0 con 0x7f7b640a3570 2026-02-25T15:01:42.349 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f73db755640 10 librados: watch_flush enter 2026-02-25T15:01:42.349 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f73db755640 10 librados: watch_flush exit 2026-02-25T15:01:42.349 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f73db755640 10 client.4477.objecter close_session for osd.4 2026-02-25T15:01:42.349 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f73db755640 1 -- 10.20.193.90:0/1294973177 >> v1:10.20.193.90:6800/1446526134 conn(0x7f73a40a34c0 legacy=0x7f73a40c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.350 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f73db755640 20 client.4477.objecter put_session s=0x7f73a40a2e10 osd=4 1 2026-02-25T15:01:42.350 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f73db755640 20 client.4477.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.350 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.347+0000 7f73db755640 10 client.4477.objecter successfully canceled tick 2026-02-25T15:01:42.350 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.349+0000 7f73db755640 1 -- 10.20.193.90:0/1294973177 >> v1:10.20.193.189:6800/3647375982 conn(0x7f73a4079fb0 legacy=0x7f73a409a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.351 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.349+0000 7f73db755640 10 monclient: shutdown 2026-02-25T15:01:42.351 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.349+0000 7f73db755640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.351 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.349+0000 7f73db755640 1 -- 10.20.193.90:0/1294973177 >> v1:10.20.193.93:6789/0 conn(0x7f73d40b9f80 legacy=0x7f73d41fc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.351 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.349+0000 7f73d9ccb640 1 -- 10.20.193.90:0/1294973177 reap_dead start 2026-02-25T15:01:42.351 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:42.351 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2026-02-25T15:01:42.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.349+0000 7f73db755640 1 -- 10.20.193.90:0/1294973177 shutdown_connections 2026-02-25T15:01:42.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.349+0000 7f73db755640 1 -- 10.20.193.90:0/1294973177 >> 10.20.193.90:0/1294973177 conn(0x7f73d40b7990 msgr2=0x7f73d40b66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.349+0000 7f73db755640 1 -- 10.20.193.90:0/1294973177 shutdown_connections 2026-02-25T15:01:42.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.349+0000 7f73db755640 1 -- 10.20.193.90:0/1294973177 wait complete. 2026-02-25T15:01:42.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.349+0000 7f73db755640 1 librados: shutdown 2026-02-25T15:01:42.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b7effd640 1 -- 10.20.193.90:0/174363358 <== osd.13 v1:10.20.193.93:6812/1850919728 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2115957317) 0x7f7b981593c0 con 0x7f7b640a3570 2026-02-25T15:01:42.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b7effd640 10 client.4483.objecter ms_dispatch2 0x7f7b98000b90 command_reply(tid 2: 0 ) 2026-02-25T15:01:42.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b7effd640 10 client.4483.objecter _finish_command 2 = system:0 2026-02-25T15:01:42.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b7effd640 20 client.4483.objecter put_session s=0x7f7b640a2ec0 osd=13 4 2026-02-25T15:01:42.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b7effd640 15 client.4483.objecter _session_command_op_remove 13 2 2026-02-25T15:01:42.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 10 librados: watch_flush enter 2026-02-25T15:01:42.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 10 librados: watch_flush exit 2026-02-25T15:01:42.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 10 client.4483.objecter close_session for osd.13 2026-02-25T15:01:42.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 1 -- 10.20.193.90:0/174363358 >> v1:10.20.193.93:6812/1850919728 conn(0x7f7b640a3570 legacy=0x7f7b640c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 20 client.4483.objecter put_session s=0x7f7b640a2ec0 osd=13 1 2026-02-25T15:01:42.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 20 client.4483.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 10 client.4483.objecter successfully canceled tick 2026-02-25T15:01:42.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 1 -- 10.20.193.90:0/174363358 >> v1:10.20.193.189:6800/3647375982 conn(0x7f7b6407a040 legacy=0x7f7b6409a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 10 monclient: shutdown 2026-02-25T15:01:42.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 1 -- 10.20.193.90:0/174363358 >> v1:10.20.193.93:6789/0 conn(0x7f7b980ba350 legacy=0x7f7b98155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b96d76640 1 -- 10.20.193.90:0/174363358 reap_dead start 2026-02-25T15:01:42.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 1 -- 10.20.193.90:0/174363358 shutdown_connections 2026-02-25T15:01:42.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 1 -- 10.20.193.90:0/174363358 >> 10.20.193.90:0/174363358 conn(0x7f7b980b7990 msgr2=0x7f7b981570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 1 -- 10.20.193.90:0/174363358 shutdown_connections 2026-02-25T15:01:42.354 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 1 -- 10.20.193.90:0/174363358 wait complete. 2026-02-25T15:01:42.354 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.350+0000 7f7b9cda2640 1 librados: shutdown 2026-02-25T15:01:42.362 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:42.362 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-25T15:01:42.362 INFO:teuthology.orchestra.run.trial090.stdout:94489280516 2026-02-25T15:01:42.362 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2026-02-25T15:01:42.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.382+0000 7fb425082640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.382+0000 7fb425082640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.382+0000 7fb425082640 10 monclient: monmap: 2026-02-25T15:01:42.383 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.384 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.384 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.383305+0000 2026-02-25T15:01:42.384 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.383305+0000 2026-02-25T15:01:42.384 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.384 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.384 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.384 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.384 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.384 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.385 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.382+0000 7f257584d640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.385 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.382+0000 7f257584d640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.385 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.382+0000 7f257584d640 10 monclient: monmap: 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.383473+0000 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.383473+0000 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7fb425082640 1 Processor -- start 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7fb425082640 1 -- start start 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7fb425082640 10 monclient: init 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7fb425082640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7fb425082640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7fb425082640 10 monclient(hunting): picked mon.noname-b con 0x7fb420157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7fb425082640 10 monclient(hunting): picked mon.noname-a con 0x7fb420157b50 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.386 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7fb425082640 10 monclient(hunting): picked mon.noname-c con 0x7fb420164d00 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7fb425082640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb42005de70 con 0x7fb420157b50 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7fb425082640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb420055b20 con 0x7fb420157780 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7fb425082640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb420055e00 con 0x7fb420164d00 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7fb425082640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7feb72c8b640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7feb72c8b640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7feb72c8b640 10 monclient: monmap: 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.384416+0000 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.384416+0000 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.387 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7f257584d640 1 Processor -- start 2026-02-25T15:01:42.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7f257584d640 1 -- start start 2026-02-25T15:01:42.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7f257584d640 10 monclient: init 2026-02-25T15:01:42.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7f257584d640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7f257584d640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:42.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7f257584d640 10 monclient(hunting): picked mon.noname-c con 0x7f2570157780 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7f257584d640 10 monclient(hunting): picked mon.noname-a con 0x7f2570157b50 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7f257584d640 10 monclient(hunting): picked mon.noname-b con 0x7f2570164d00 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7f257584d640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f257005de70 con 0x7f2570157b50 2026-02-25T15:01:42.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7f257584d640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2570055b20 con 0x7f2570164d00 2026-02-25T15:01:42.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7f257584d640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2570055e00 con 0x7f2570157780 2026-02-25T15:01:42.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.383+0000 7f257584d640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7f257584d640 10 monclient(hunting): authenticate will time out at 2991.984009s 2026-02-25T15:01:42.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7fb425082640 10 monclient(hunting): authenticate will time out at 2991.984228s 2026-02-25T15:01:42.389 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7f256f7fe640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f2570164d00 0x7f25701c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46562/0 (socket says 10.20.193.90:46562) 2026-02-25T15:01:42.389 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7f256f7fe640 1 -- 10.20.193.90:0/1172055372 learned_addr learned my addr 10.20.193.90:0/1172055372 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.389 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7fb41f577640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fb420164d00 0x7fb4201c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54286/0 (socket says 10.20.193.90:54286) 2026-02-25T15:01:42.389 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7fb41f577640 1 -- 10.20.193.90:0/204199882 learned_addr learned my addr 10.20.193.90:0/204199882 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.389 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7feb72c8b640 1 Processor -- start 2026-02-25T15:01:42.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7feb72c8b640 1 -- start start 2026-02-25T15:01:42.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.384+0000 7feb72c8b640 10 monclient: init 2026-02-25T15:01:42.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.385+0000 7feb72c8b640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.385+0000 7feb72c8b640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:42.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.385+0000 7feb72c8b640 10 monclient(hunting): picked mon.noname-b con 0x7feb6c0b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.385+0000 7feb72c8b640 10 monclient(hunting): picked mon.noname-c con 0x7feb6c0ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.385+0000 7feb72c8b640 10 monclient(hunting): picked mon.noname-a con 0x7feb6c0585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.385+0000 7feb72c8b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c163930 con 0x7feb6c0585e0 2026-02-25T15:01:42.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.385+0000 7feb72c8b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c163bb0 con 0x7feb6c0b9f80 2026-02-25T15:01:42.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.385+0000 7feb72c8b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c163e90 con 0x7feb6c0ba350 2026-02-25T15:01:42.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.385+0000 7feb72c8b640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.385+0000 7feb72c8b640 10 monclient(hunting): authenticate will time out at 2991.985291s 2026-02-25T15:01:42.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.386+0000 7feb70a00640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7feb6c0b9f80 0x7feb6c163220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46586/0 (socket says 10.20.193.90:46586) 2026-02-25T15:01:42.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.386+0000 7feb70a00640 1 -- 10.20.193.90:0/3601450384 learned_addr learned my addr 10.20.193.90:0/3601450384 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.386+0000 7f256dffb640 1 -- 10.20.193.90:0/1172055372 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4197003248 0 0) 0x7f2570055b20 con 0x7f2570164d00 2026-02-25T15:01:42.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.386+0000 7f256dffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.386+0000 7f256dffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.386+0000 7f256dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.386+0000 7f256dffb640 10 monclient(hunting): my global_id is 4492 2026-02-25T15:01:42.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.386+0000 7f256dffb640 1 -- 10.20.193.90:0/1172055372 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f255c003610 con 0x7f2570164d00 2026-02-25T15:01:42.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.387+0000 7fb41dd74640 1 -- 10.20.193.90:0/204199882 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1342333931 0 0) 0x7fb420055e00 con 0x7fb420164d00 2026-02-25T15:01:42.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.387+0000 7fb41dd74640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.387+0000 7fb41dd74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.387+0000 7fb41dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.387+0000 7fb41dd74640 10 monclient(hunting): my global_id is 4484 2026-02-25T15:01:42.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.387+0000 7fb41dd74640 1 -- 10.20.193.90:0/204199882 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb40c003610 con 0x7fb420164d00 2026-02-25T15:01:42.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.389+0000 7fb41dd74640 1 -- 10.20.193.90:0/204199882 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 387345358 0 0) 0x7fb40c003610 con 0x7fb420164d00 2026-02-25T15:01:42.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.389+0000 7fb41dd74640 1 -- 10.20.193.90:0/204199882 >> v1:10.20.193.93:6789/0 conn(0x7fb420157780 legacy=0x7fb420056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.389+0000 7fb41dd74640 1 -- 10.20.193.90:0/204199882 >> v1:10.20.193.90:6789/0 conn(0x7fb420157b50 legacy=0x7fb4201645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.389+0000 7fb41dd74640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.389+0000 7fb41dd74640 1 monclient: found mon.noname-c 2026-02-25T15:01:42.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.389+0000 7fb41dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.389+0000 7fb41dd74640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.389+0000 7fb41dd74640 1 -- 10.20.193.90:0/204199882 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4200560e0 con 0x7fb420164d00 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.389+0000 7fb41dd74640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.389+0000 7fb41dd74640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.389+0000 7fb41dd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7fb41dd74640 1 -- 10.20.193.90:0/204199882 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb410003170 con 0x7fb420164d00 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7fb41dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7fb41dd74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7fb41dd74640 10 monclient: dump: 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7fb425082640 5 monclient: authenticate success, global_id 4484 2026-02-25T15:01:42.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7fb425082640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f256dffb640 1 -- 10.20.193.90:0/1172055372 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3121922917 0 0) 0x7f255c003610 con 0x7f2570164d00 2026-02-25T15:01:42.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f256dffb640 1 -- 10.20.193.90:0/1172055372 >> v1:10.20.193.94:6789/0 conn(0x7f2570157780 legacy=0x7f2570056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f256dffb640 1 -- 10.20.193.90:0/1172055372 >> v1:10.20.193.90:6789/0 conn(0x7f2570157b50 legacy=0x7f25701645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f256dffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f256dffb640 1 monclient: found mon.noname-b 2026-02-25T15:01:42.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7feb637fe640 1 -- 10.20.193.90:0/3601450384 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2920976611 0 0) 0x7feb6c163bb0 con 0x7feb6c0b9f80 2026-02-25T15:01:42.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7feb637fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7feb637fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7feb637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7feb637fe640 10 monclient(hunting): my global_id is 4498 2026-02-25T15:01:42.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7feb637fe640 1 -- 10.20.193.90:0/3601450384 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb58003610 con 0x7feb6c0b9f80 2026-02-25T15:01:42.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f256dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f256dffb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f256dffb640 1 -- 10.20.193.90:0/1172055372 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25700560e0 con 0x7f2570164d00 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f256dffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f256dffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f256dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f256dffb640 1 -- 10.20.193.90:0/1172055372 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2560003170 con 0x7f2570164d00 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f256dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f256dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f256dffb640 10 monclient: dump: 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f257584d640 5 monclient: authenticate success, global_id 4492 2026-02-25T15:01:42.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.390+0000 7f257584d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.392+0000 7feb637fe640 1 -- 10.20.193.90:0/3601450384 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 254640052 0 0) 0x7feb68002c90 con 0x7feb6c0585e0 2026-02-25T15:01:42.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.392+0000 7feb637fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.392+0000 7feb637fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.392+0000 7feb637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.392+0000 7feb637fe640 10 monclient(hunting): my global_id is 4476 2026-02-25T15:01:42.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.392+0000 7feb637fe640 1 -- 10.20.193.90:0/3601450384 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb6c163bb0 con 0x7feb6c0585e0 2026-02-25T15:01:42.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.392+0000 7feb637fe640 1 -- 10.20.193.90:0/3601450384 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1086075596 0 0) 0x7feb6c163e90 con 0x7feb6c0ba350 2026-02-25T15:01:42.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.392+0000 7feb637fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.392+0000 7feb637fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.392+0000 7feb637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.392+0000 7feb637fe640 10 monclient(hunting): my global_id is 4490 2026-02-25T15:01:42.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.392+0000 7feb637fe640 1 -- 10.20.193.90:0/3601450384 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb68002c90 con 0x7feb6c0ba350 2026-02-25T15:01:42.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7fb41dd74640 1 -- 10.20.193.90:0/204199882 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb410003380 con 0x7fb420164d00 2026-02-25T15:01:42.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7fb41dd74640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7fb41dd74640 1 -- 10.20.193.90:0/204199882 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb4100037c0 con 0x7fb420164d00 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7fb41dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7fb41dd74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7fb41dd74640 10 monclient: dump: 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7fb425082640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7fb425082640 4 set_mon_vals no callback set 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7f9131999640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7f9131999640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7f9131999640 10 monclient: monmap: 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.394034+0000 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.394034+0000 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7f256dffb640 1 -- 10.20.193.90:0/1172055372 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2560003380 con 0x7f2570164d00 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7f256dffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7f256dffb640 1 -- 10.20.193.90:0/1172055372 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f25600037c0 con 0x7f2570164d00 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7f256dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7f256dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7f256dffb640 10 monclient: dump: 2026-02-25T15:01:42.399 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.399 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.399 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.399 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.399 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.399 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.399 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.399 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.399 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.399 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7f257584d640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7f257584d640 4 set_mon_vals no callback set 2026-02-25T15:01:42.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb637fe640 1 -- 10.20.193.90:0/3601450384 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3600817293 0 0) 0x7feb58003610 con 0x7feb6c0b9f80 2026-02-25T15:01:42.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb637fe640 1 -- 10.20.193.90:0/3601450384 >> v1:10.20.193.94:6789/0 conn(0x7feb6c0ba350 legacy=0x7feb6c057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb637fe640 1 -- 10.20.193.90:0/3601450384 >> v1:10.20.193.90:6789/0 conn(0x7feb6c0585e0 legacy=0x7feb6c166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb637fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb637fe640 1 monclient: found mon.noname-b 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb637fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb637fe640 1 -- 10.20.193.90:0/3601450384 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c0573f0 con 0x7feb6c0b9f80 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb637fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb637fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb637fe640 1 -- 10.20.193.90:0/3601450384 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7feb5c002df0 con 0x7feb6c0b9f80 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb637fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb637fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb637fe640 10 monclient: dump: 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.400 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb72c8b640 5 monclient: authenticate success, global_id 4498 2026-02-25T15:01:42.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7fb425082640 10 monclient: shutdown 2026-02-25T15:01:42.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7fb425082640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7fb425082640 1 -- 10.20.193.90:0/204199882 >> v1:10.20.193.94:6789/0 conn(0x7fb420164d00 legacy=0x7fb4201c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7feb72c8b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7fb425082640 1 -- 10.20.193.90:0/204199882 shutdown_connections 2026-02-25T15:01:42.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7fb425082640 1 -- 10.20.193.90:0/204199882 >> 10.20.193.90:0/204199882 conn(0x7fb4200b2ef0 msgr2=0x7fb4200b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7fb425082640 1 -- 10.20.193.90:0/204199882 shutdown_connections 2026-02-25T15:01:42.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.393+0000 7fb425082640 1 -- 10.20.193.90:0/204199882 wait complete. 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7f257584d640 10 monclient: shutdown 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7f257584d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7f257584d640 1 -- 10.20.193.90:0/1172055372 >> v1:10.20.193.93:6789/0 conn(0x7f2570164d00 legacy=0x7f25701c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7faffee7b640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7faffee7b640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7faffee7b640 10 monclient: monmap: 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.394688+0000 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.394688+0000 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.402 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7f257584d640 1 -- 10.20.193.90:0/1172055372 shutdown_connections 2026-02-25T15:01:42.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7f257584d640 1 -- 10.20.193.90:0/1172055372 >> 10.20.193.90:0/1172055372 conn(0x7f25700b2ef0 msgr2=0x7f25700b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7f257584d640 1 -- 10.20.193.90:0/1172055372 shutdown_connections 2026-02-25T15:01:42.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7fb425082640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7fb425082640 10 monclient: monmap: 2026-02-25T15:01:42.403 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.403 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.403 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.403 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.403 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.403 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.403 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.403 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.403 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.403 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7faffee7b640 1 Processor -- start 2026-02-25T15:01:42.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7fb425082640 1 librados: starting msgr at 2026-02-25T15:01:42.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7fb425082640 1 librados: starting objecter 2026-02-25T15:01:42.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7faffee7b640 1 -- start start 2026-02-25T15:01:42.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7faffee7b640 10 monclient: init 2026-02-25T15:01:42.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7fb425082640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7feb637fe640 1 -- 10.20.193.90:0/3601450384 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7feb5c002fc0 con 0x7feb6c0b9f80 2026-02-25T15:01:42.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7feb637fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7feb637fe640 1 -- 10.20.193.90:0/3601450384 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7feb5c003400 con 0x7feb6c0b9f80 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7feb637fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7feb637fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7feb637fe640 10 monclient: dump: 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7feb72c8b640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7feb72c8b640 4 set_mon_vals no callback set 2026-02-25T15:01:42.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7fb425082640 1 Processor -- start 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb425082640 1 -- start start 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb425082640 1 librados: setting wanted keys 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb425082640 1 librados: calling monclient init 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb425082640 10 monclient: init 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb425082640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb425082640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb425082640 10 monclient(hunting): picked mon.noname-b con 0x7fb420157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb425082640 10 monclient(hunting): picked mon.noname-c con 0x7fb420157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb425082640 10 monclient(hunting): picked mon.noname-a con 0x7fb420164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb425082640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4200af7b0 con 0x7fb420164d00 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb425082640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4200a80c0 con 0x7fb420157780 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb425082640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4200a83a0 con 0x7fb420157b50 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb425082640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb425082640 10 monclient(hunting): authenticate will time out at 2991.995583s 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7feb72c8b640 10 monclient: shutdown 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7feb72c8b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7feb72c8b640 1 -- 10.20.193.90:0/3601450384 >> v1:10.20.193.93:6789/0 conn(0x7feb6c0b9f80 legacy=0x7feb6c163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7f257584d640 1 -- 10.20.193.90:0/1172055372 wait complete. 2026-02-25T15:01:42.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7faffee7b640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7faffee7b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:42.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7faffee7b640 10 monclient(hunting): picked mon.noname-a con 0x7faff80b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7faffee7b640 10 monclient(hunting): picked mon.noname-c con 0x7faff80ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7faffee7b640 10 monclient(hunting): picked mon.noname-b con 0x7faff80585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7faffee7b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff8163930 con 0x7faff80b9f80 2026-02-25T15:01:42.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7faffee7b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff8163bb0 con 0x7faff80585e0 2026-02-25T15:01:42.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7faffee7b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff8163e90 con 0x7faff80ba350 2026-02-25T15:01:42.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7faffee7b640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7faffee7b640 10 monclient(hunting): authenticate will time out at 2991.995789s 2026-02-25T15:01:42.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb41ed76640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fb420157780 0x7fb4200a9440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46600/0 (socket says 10.20.193.90:46600) 2026-02-25T15:01:42.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7fb41ed76640 1 -- 10.20.193.90:0/2259846989 learned_addr learned my addr 10.20.193.90:0/2259846989 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7f9131999640 1 Processor -- start 2026-02-25T15:01:42.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7f9131999640 1 -- start start 2026-02-25T15:01:42.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.394+0000 7f9131999640 10 monclient: init 2026-02-25T15:01:42.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7f9131999640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7f9131999640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:42.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7f9131999640 10 monclient(hunting): picked mon.noname-c con 0x7f912c155b20 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7f9131999640 10 monclient(hunting): picked mon.noname-a con 0x7f912c056c70 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7f9131999640 10 monclient(hunting): picked mon.noname-b con 0x7f912c162f10 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7f9131999640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f912c0b18f0 con 0x7f912c056c70 2026-02-25T15:01:42.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7f9131999640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f912c0b1bd0 con 0x7f912c162f10 2026-02-25T15:01:42.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7f9131999640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f912c0b1eb0 con 0x7f912c155b20 2026-02-25T15:01:42.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7f9131999640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7f9131999640 10 monclient(hunting): authenticate will time out at 2991.995541s 2026-02-25T15:01:42.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7feb72c8b640 1 -- 10.20.193.90:0/3601450384 shutdown_connections 2026-02-25T15:01:42.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7feb72c8b640 1 -- 10.20.193.90:0/3601450384 >> 10.20.193.90:0/3601450384 conn(0x7feb6c0b7990 msgr2=0x7feb6c0b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7f257584d640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.395+0000 7f257584d640 10 monclient: monmap: 2026-02-25T15:01:42.409 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.409 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.409 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.409 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.409 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.409 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.409 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.409 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.409 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.409 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7feb72c8b640 1 -- 10.20.193.90:0/3601450384 shutdown_connections 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f257584d640 1 librados: starting msgr at 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f257584d640 1 librados: starting objecter 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f257584d640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f257584d640 1 Processor -- start 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f44d6604640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f44d6604640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f44d6604640 10 monclient: monmap: 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.396852+0000 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.396852+0000 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f912a7fc640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f912c056c70 0x7f912c162800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45068/0 (socket says 10.20.193.90:45068) 2026-02-25T15:01:42.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f912a7fc640 1 -- 10.20.193.90:0/3114250905 learned_addr learned my addr 10.20.193.90:0/3114250905 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7feb72c8b640 1 -- 10.20.193.90:0/3601450384 wait complete. 2026-02-25T15:01:42.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f257584d640 1 -- start start 2026-02-25T15:01:42.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f257584d640 1 librados: setting wanted keys 2026-02-25T15:01:42.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f257584d640 1 librados: calling monclient init 2026-02-25T15:01:42.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f257584d640 10 monclient: init 2026-02-25T15:01:42.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7faffcbf0640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7faff80b9f80 0x7faff8163220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45080/0 (socket says 10.20.193.90:45080) 2026-02-25T15:01:42.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7fb3ff7fe640 10 client.?.objecter ms_handle_connect 0x7fb420157780 2026-02-25T15:01:42.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7fb3ff7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7faffcbf0640 1 -- 10.20.193.90:0/3518157365 learned_addr learned my addr 10.20.193.90:0/3518157365 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7feb72c8b640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7feb72c8b640 10 monclient: monmap: 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f257584d640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f257584d640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f257584d640 10 monclient(hunting): picked mon.noname-c con 0x7f2570157780 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f257584d640 10 monclient(hunting): picked mon.noname-b con 0x7f2570157b50 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f257584d640 10 monclient(hunting): picked mon.noname-a con 0x7f2570164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7f257584d640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2570156a80 con 0x7f2570164d00 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7f257584d640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f257013c0a0 con 0x7f2570157b50 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7f257584d640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f257013c380 con 0x7f2570157780 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7f257584d640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7f257584d640 10 monclient(hunting): authenticate will time out at 2991.997052s 2026-02-25T15:01:42.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f44d6604640 1 Processor -- start 2026-02-25T15:01:42.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f44d6604640 1 -- start start 2026-02-25T15:01:42.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.396+0000 7f44d6604640 10 monclient: init 2026-02-25T15:01:42.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7feb72c8b640 1 librados: starting msgr at 2026-02-25T15:01:42.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7fb3ff7fe640 10 client.?.objecter ms_handle_connect 0x7fb420157b50 2026-02-25T15:01:42.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7fb3ff7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7feb72c8b640 1 librados: starting objecter 2026-02-25T15:01:42.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7feb72c8b640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7feb72c8b640 1 Processor -- start 2026-02-25T15:01:42.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7f44d6604640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7feb72c8b640 1 -- start start 2026-02-25T15:01:42.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7feb72c8b640 1 librados: setting wanted keys 2026-02-25T15:01:42.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7feb72c8b640 1 librados: calling monclient init 2026-02-25T15:01:42.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7feb72c8b640 10 monclient: init 2026-02-25T15:01:42.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7f44d6604640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:42.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7f44d6604640 10 monclient(hunting): picked mon.noname-c con 0x7f44d0059e70 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7f44d6604640 10 monclient(hunting): picked mon.noname-b con 0x7f44d005a240 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7f44d6604640 10 monclient(hunting): picked mon.noname-a con 0x7f44d0162e00 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7f44d6604640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d00b3210 con 0x7f44d0162e00 2026-02-25T15:01:42.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7f44d6604640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d00b23b0 con 0x7f44d005a240 2026-02-25T15:01:42.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7fb3ff7fe640 10 client.?.objecter ms_handle_connect 0x7fb420164d00 2026-02-25T15:01:42.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7fb3ff7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7feb72c8b640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7feb72c8b640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:42.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7feb72c8b640 10 monclient(hunting): picked mon.noname-c con 0x7feb6c0585e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7feb72c8b640 10 monclient(hunting): picked mon.noname-b con 0x7feb6c0b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7feb72c8b640 10 monclient(hunting): picked mon.noname-a con 0x7feb6c0ba350 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7feb72c8b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c166030 con 0x7feb6c0ba350 2026-02-25T15:01:42.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7feb72c8b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c1ff490 con 0x7feb6c0b9f80 2026-02-25T15:01:42.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7feb72c8b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c1ff770 con 0x7feb6c0585e0 2026-02-25T15:01:42.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7feb72c8b640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7feb72c8b640 10 monclient(hunting): authenticate will time out at 2991.998540s 2026-02-25T15:01:42.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7f44d6604640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d00b2690 con 0x7f44d0059e70 2026-02-25T15:01:42.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7f44d6604640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.397+0000 7f44d6604640 10 monclient(hunting): authenticate will time out at 2991.997670s 2026-02-25T15:01:42.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 252322248 0 0) 0x7fb4200a80c0 con 0x7fb420157780 2026-02-25T15:01:42.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7fb3ff7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7fb3ff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7fb3ff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7fb3ff7fe640 10 monclient(hunting): my global_id is 4501 2026-02-25T15:01:42.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.398+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3ec003610 con 0x7fb420157780 2026-02-25T15:01:42.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.399+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3394958658 0 0) 0x7fb4200a83a0 con 0x7fb420157b50 2026-02-25T15:01:42.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.399+0000 7fb3ff7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.399+0000 7fb3ff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.399+0000 7f256effd640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f2570157780 0x7f2570156370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54336/0 (socket says 10.20.193.90:54336) 2026-02-25T15:01:42.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.399+0000 7f256effd640 1 -- 10.20.193.90:0/359940460 learned_addr learned my addr 10.20.193.90:0/359940460 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.399+0000 7fb3ff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.399+0000 7fb3ff7fe640 10 monclient(hunting): my global_id is 4493 2026-02-25T15:01:42.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.399+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4200a80c0 con 0x7fb420157b50 2026-02-25T15:01:42.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.399+0000 7f44cffff640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f44d0059e70 0x7f44d00b1a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54338/0 (socket says 10.20.193.90:54338) 2026-02-25T15:01:42.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.399+0000 7f44cffff640 1 -- 10.20.193.90:0/3403664117 learned_addr learned my addr 10.20.193.90:0/3403664117 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.399+0000 7f2547fff640 10 client.?.objecter ms_handle_connect 0x7f2570157780 2026-02-25T15:01:42.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.399+0000 7f2547fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7f2547fff640 10 client.?.objecter ms_handle_connect 0x7f2570164d00 2026-02-25T15:01:42.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7f2547fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7fafef7fe640 1 -- 10.20.193.90:0/3518157365 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1172393264 0 0) 0x7faff8163bb0 con 0x7faff80585e0 2026-02-25T15:01:42.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7fafef7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7fafef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7fafef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7fafef7fe640 10 monclient(hunting): my global_id is 4504 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7fafef7fe640 1 -- 10.20.193.90:0/3518157365 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafe4003610 con 0x7faff80585e0 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7f4d4f71a640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7f4d4f71a640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7f4d4f71a640 10 monclient: monmap: 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.401419+0000 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.401419+0000 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7f9129ffb640 1 -- 10.20.193.90:0/3114250905 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1341929364 0 0) 0x7f912c0b1bd0 con 0x7f912c162f10 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7f9129ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7f9129ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7f9129ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7f9129ffb640 10 monclient(hunting): my global_id is 4507 2026-02-25T15:01:42.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7f9129ffb640 1 -- 10.20.193.90:0/3114250905 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9118003610 con 0x7f912c162f10 2026-02-25T15:01:42.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7fafef7fe640 1 -- 10.20.193.90:0/3518157365 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1756084792 0 0) 0x7fafe8002910 con 0x7faff80b9f80 2026-02-25T15:01:42.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7fafef7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7fafef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7fafef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7fafef7fe640 10 monclient(hunting): my global_id is 4482 2026-02-25T15:01:42.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.400+0000 7fafef7fe640 1 -- 10.20.193.90:0/3518157365 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faff8163bb0 con 0x7faff80b9f80 2026-02-25T15:01:42.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.401+0000 7f2547fff640 10 client.?.objecter ms_handle_connect 0x7f2570157b50 2026-02-25T15:01:42.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.401+0000 7f2547fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.401+0000 7feb63fff640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7feb6c0b9f80 0x7feb6c165920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46650/0 (socket says 10.20.193.90:46650) 2026-02-25T15:01:42.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.401+0000 7f4d4f71a640 1 Processor -- start 2026-02-25T15:01:42.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.401+0000 7feb63fff640 1 -- 10.20.193.90:0/3231812946 learned_addr learned my addr 10.20.193.90:0/3231812946 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.401+0000 7f4d4f71a640 1 -- start start 2026-02-25T15:01:42.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.401+0000 7f4d4f71a640 10 monclient: init 2026-02-25T15:01:42.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f4d4f71a640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f4d4f71a640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:42.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f9129ffb640 1 -- 10.20.193.90:0/3114250905 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 139542973 0 0) 0x7f912c0b1eb0 con 0x7f912c155b20 2026-02-25T15:01:42.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f9129ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f9129ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f4d4f71a640 10 monclient(hunting): picked mon.noname-b con 0x7f4d48159a70 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f9129ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f9129ffb640 10 monclient(hunting): my global_id is 4496 2026-02-25T15:01:42.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f9129ffb640 1 -- 10.20.193.90:0/3114250905 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f912c0b1bd0 con 0x7f912c155b20 2026-02-25T15:01:42.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f4d4f71a640 10 monclient(hunting): picked mon.noname-c con 0x7f4d48159e40 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f4d4f71a640 10 monclient(hunting): picked mon.noname-a con 0x7f4d48058070 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f4d4f71a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d48163930 con 0x7f4d48058070 2026-02-25T15:01:42.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f4d4f71a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d48163bf0 con 0x7f4d48159a70 2026-02-25T15:01:42.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f4d4f71a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d48163ed0 con 0x7f4d48159e40 2026-02-25T15:01:42.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f4d4f71a640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f4d4f71a640 10 monclient(hunting): authenticate will time out at 2992.002576s 2026-02-25T15:01:42.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7feb617fa640 10 client.?.objecter ms_handle_connect 0x7feb6c0b9f80 2026-02-25T15:01:42.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2232521977 0 0) 0x7f257013c380 con 0x7f2570157780 2026-02-25T15:01:42.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f2547fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f2547fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f2547fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.402+0000 7f2547fff640 10 monclient(hunting): my global_id is 4505 2026-02-25T15:01:42.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7feb617fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2540003610 con 0x7f2570157780 2026-02-25T15:01:42.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7f44ceffd640 1 -- 10.20.193.90:0/3403664117 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3812995019 0 0) 0x7f44d00b2690 con 0x7f44d0059e70 2026-02-25T15:01:42.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7f44ceffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7f44ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7f44ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7f44ceffd640 10 monclient(hunting): my global_id is 4502 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7f44ceffd640 1 -- 10.20.193.90:0/3403664117 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44b4003610 con 0x7f44d0059e70 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2930980918 0 0) 0x7fb4200af7b0 con 0x7fb420164d00 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7fb3ff7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7fb3ff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7fb3ff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7f9129ffb640 1 -- 10.20.193.90:0/3114250905 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2628722400 0 0) 0x7f912c0b18f0 con 0x7f912c056c70 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7fb3ff7fe640 10 monclient(hunting): my global_id is 4485 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7f9129ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7f9129ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7f9129ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7f9129ffb640 10 monclient(hunting): my global_id is 4479 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7f9129ffb640 1 -- 10.20.193.90:0/3114250905 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f912c0b1eb0 con 0x7f912c056c70 2026-02-25T15:01:42.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.403+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4200a83a0 con 0x7fb420164d00 2026-02-25T15:01:42.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.405+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3350657672 0 0) 0x7f2570156a80 con 0x7f2570164d00 2026-02-25T15:01:42.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.405+0000 7feb617fa640 10 client.?.objecter ms_handle_connect 0x7feb6c0585e0 2026-02-25T15:01:42.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.405+0000 7feb617fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.405+0000 7f2547fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.405+0000 7f2547fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.405+0000 7f2547fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.405+0000 7f2547fff640 10 monclient(hunting): my global_id is 4488 2026-02-25T15:01:42.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.405+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f257013c380 con 0x7f2570164d00 2026-02-25T15:01:42.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.405+0000 7feb617fa640 10 client.?.objecter ms_handle_connect 0x7feb6c0ba350 2026-02-25T15:01:42.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.405+0000 7feb617fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.406+0000 7fcca0ee8640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.406+0000 7fcca0ee8640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.406+0000 7fcca0ee8640 10 monclient: monmap: 2026-02-25T15:01:42.423 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.423 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.407031+0000 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.407031+0000 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.406+0000 7f0df5171640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.406+0000 7f0df5171640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.406+0000 7f0df5171640 10 monclient: monmap: 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.407476+0000 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.407476+0000 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.406+0000 7fcca0ee8640 1 Processor -- start 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fcca0ee8640 1 -- start start 2026-02-25T15:01:42.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fcca0ee8640 10 monclient: init 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fafef7fe640 1 -- 10.20.193.90:0/3518157365 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1516597604 0 0) 0x7fafe4003610 con 0x7faff80585e0 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fafef7fe640 1 -- 10.20.193.90:0/3518157365 >> v1:10.20.193.94:6789/0 conn(0x7faff80ba350 legacy=0x7faff8057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fafef7fe640 1 -- 10.20.193.90:0/3518157365 >> v1:10.20.193.90:6789/0 conn(0x7faff80b9f80 legacy=0x7faff8163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fafef7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fafef7fe640 1 monclient: found mon.noname-b 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fafef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fafef7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fafef7fe640 1 -- 10.20.193.90:0/3518157365 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff80573f0 con 0x7faff80585e0 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fafef7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fafef7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fafef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fafef7fe640 1 -- 10.20.193.90:0/3518157365 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7faff4003170 con 0x7faff80585e0 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fafef7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fafef7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fafef7fe640 10 monclient: dump: 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7faffee7b640 5 monclient: authenticate success, global_id 4504 2026-02-25T15:01:42.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7faffee7b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7f0df5171640 1 Processor -- start 2026-02-25T15:01:42.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7f0df5171640 1 -- start start 2026-02-25T15:01:42.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7f0df5171640 10 monclient: init 2026-02-25T15:01:42.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fcca0ee8640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fcca0ee8640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:42.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fcca0ee8640 10 monclient(hunting): picked mon.noname-c con 0x7fcc9c0b9f80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fcca0ee8640 10 monclient(hunting): picked mon.noname-b con 0x7fcc9c0ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7f4d4dc90640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f4d48058070 0x7f4d48058460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45122/0 (socket says 10.20.193.90:45122) 2026-02-25T15:01:42.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7f4d4dc90640 1 -- 10.20.193.90:0/2747018121 learned_addr learned my addr 10.20.193.90:0/2747018121 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fcca0ee8640 10 monclient(hunting): picked mon.noname-a con 0x7fcc9c0585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fcca0ee8640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc9c163930 con 0x7fcc9c0585e0 2026-02-25T15:01:42.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fcca0ee8640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc9c163bb0 con 0x7fcc9c0ba350 2026-02-25T15:01:42.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fcca0ee8640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc9c163e90 con 0x7fcc9c0b9f80 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fcca0ee8640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.407+0000 7fcca0ee8640 10 monclient(hunting): authenticate will time out at 2992.007944s 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.408+0000 7f0df5171640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.408+0000 7f0df5171640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.408+0000 7f0df5171640 10 monclient(hunting): picked mon.noname-c con 0x7f0df015b0a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.408+0000 7f0df5171640 10 monclient(hunting): picked mon.noname-a con 0x7f0df00b3190 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.408+0000 7f0df5171640 10 monclient(hunting): picked mon.noname-b con 0x7f0df00b5da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.408+0000 7f0df5171640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df00b3610 con 0x7f0df00b3190 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.408+0000 7f789b18a640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.408+0000 7f789b18a640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.408+0000 7f789b18a640 10 monclient: monmap: 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.409397+0000 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.409397+0000 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.427 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.408+0000 7f0df5171640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df00b38f0 con 0x7f0df00b5da0 2026-02-25T15:01:42.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.408+0000 7f0df5171640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df00b3bd0 con 0x7f0df015b0a0 2026-02-25T15:01:42.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.408+0000 7f0df5171640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.408+0000 7f0df5171640 10 monclient(hunting): authenticate will time out at 2992.008858s 2026-02-25T15:01:42.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3556907151 0 0) 0x7f257013c0a0 con 0x7f2570157b50 2026-02-25T15:01:42.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f2547fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f2547fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f2547fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f2547fff640 10 monclient(hunting): my global_id is 4510 2026-02-25T15:01:42.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2570156a80 con 0x7f2570157b50 2026-02-25T15:01:42.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f789b18a640 1 Processor -- start 2026-02-25T15:01:42.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f44ceffd640 1 -- 10.20.193.90:0/3403664117 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1995834783 0 0) 0x7f44d00b3210 con 0x7f44d0162e00 2026-02-25T15:01:42.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f44ceffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f44ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f44ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f44ceffd640 10 monclient(hunting): my global_id is 4491 2026-02-25T15:01:42.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f44ceffd640 1 -- 10.20.193.90:0/3403664117 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44d00b2690 con 0x7f44d0162e00 2026-02-25T15:01:42.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f789b18a640 1 -- start start 2026-02-25T15:01:42.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3877673311 0 0) 0x7feb6c1ff490 con 0x7feb6c0b9f80 2026-02-25T15:01:42.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7feb617fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7feb617fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7feb617fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7feb617fa640 10 monclient(hunting): my global_id is 4513 2026-02-25T15:01:42.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f789b18a640 10 monclient: init 2026-02-25T15:01:42.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb3c003610 con 0x7feb6c0b9f80 2026-02-25T15:01:42.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f789b18a640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f789b18a640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:42.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f789b18a640 10 monclient(hunting): picked mon.noname-b con 0x7f78940b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f789b18a640 10 monclient(hunting): picked mon.noname-c con 0x7f78940ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9129ffb640 1 -- 10.20.193.90:0/3114250905 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3586351654 0 0) 0x7f9118003610 con 0x7f912c162f10 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9129ffb640 1 -- 10.20.193.90:0/3114250905 >> v1:10.20.193.94:6789/0 conn(0x7f912c155b20 legacy=0x7f912c0b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9129ffb640 1 -- 10.20.193.90:0/3114250905 >> v1:10.20.193.90:6789/0 conn(0x7f912c056c70 legacy=0x7f912c162800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9129ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9129ffb640 1 monclient: found mon.noname-b 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f789b18a640 10 monclient(hunting): picked mon.noname-a con 0x7f78940585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9129ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9129ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9129ffb640 1 -- 10.20.193.90:0/3114250905 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c0b2190 con 0x7f912c162f10 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9129ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9129ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9129ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9129ffb640 1 -- 10.20.193.90:0/3114250905 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f911c003170 con 0x7f912c162f10 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9129ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9129ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9129ffb640 10 monclient: dump: 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9131999640 5 monclient: authenticate success, global_id 4507 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f789b18a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7894163930 con 0x7f78940585e0 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7f9131999640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7fcc9ad76640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fcc9c0585e0 0x7fcc9c166cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45136/0 (socket says 10.20.193.90:45136) 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7fcc9ad76640 1 -- 10.20.193.90:0/1911448125 learned_addr learned my addr 10.20.193.90:0/1911448125 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7f789b18a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7894163bb0 con 0x7f78940b9f80 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7f789b18a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7894163e90 con 0x7f78940ba350 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7f789b18a640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7f789b18a640 10 monclient(hunting): authenticate will time out at 2992.010046s 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7f44ceffd640 1 -- 10.20.193.90:0/3403664117 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1138181703 0 0) 0x7f44d00b23b0 con 0x7f44d005a240 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4164884625 0 0) 0x7fb3ec003610 con 0x7fb420157780 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb4200af7b0 con 0x7fb420157780 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb414003100 con 0x7fb420157780 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7f44ceffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7f44ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7f44ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7f44ceffd640 10 monclient(hunting): my global_id is 4516 2026-02-25T15:01:42.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7f44ceffd640 1 -- 10.20.193.90:0/3403664117 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44d00b3210 con 0x7f44d005a240 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7fb3ff7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7fb3ff7fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7fb3ff7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.409+0000 7fb3ff7fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3693534406 0 0) 0x7fb4200a80c0 con 0x7fb420157b50 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb3ec003610 con 0x7fb420157b50 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb408003090 con 0x7fb420157b50 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7fb3ff7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7fb3ff7fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.410+0000 7fb3ff7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.432 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f461167c640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f461167c640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f461167c640 10 monclient: monmap: 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.412682+0000 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.412682+0000 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44ceffd640 1 -- 10.20.193.90:0/3403664117 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 240931825 0 0) 0x7f44b4003610 con 0x7f44d0059e70 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44ceffd640 1 -- 10.20.193.90:0/3403664117 >> v1:10.20.193.93:6789/0 conn(0x7f44d005a240 legacy=0x7f44d01626f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44ceffd640 1 -- 10.20.193.90:0/3403664117 >> v1:10.20.193.90:6789/0 conn(0x7f44d0162e00 legacy=0x7f44d01613e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44ceffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44ceffd640 1 monclient: found mon.noname-c 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44ceffd640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44ceffd640 1 -- 10.20.193.90:0/3403664117 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d00b2970 con 0x7f44d0059e70 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44ceffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44ceffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44ceffd640 1 -- 10.20.193.90:0/3403664117 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f44bc002df0 con 0x7f44d0059e70 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44ceffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44ceffd640 10 monclient: dump: 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.433 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44d6604640 5 monclient: authenticate success, global_id 4502 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f44d6604640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f461167c640 1 Processor -- start 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f461167c640 1 -- start start 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.412+0000 7f461167c640 10 monclient: init 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f461167c640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f461167c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f461167c640 10 monclient(hunting): picked mon.noname-c con 0x7f460c0b9f80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f461167c640 10 monclient(hunting): picked mon.noname-a con 0x7f460c0ba350 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f461167c640 10 monclient(hunting): picked mon.noname-b con 0x7f460c0585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f461167c640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f460c163930 con 0x7f460c0ba350 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f461167c640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f460c163bb0 con 0x7f460c0585e0 2026-02-25T15:01:42.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f461167c640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f460c163e90 con 0x7f460c0b9f80 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f461167c640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f461167c640 10 monclient(hunting): authenticate will time out at 2992.013368s 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4196652364 0 0) 0x7feb6c1ff770 con 0x7feb6c0585e0 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7feb617fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7feb617fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7feb617fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7feb617fa640 10 monclient(hunting): my global_id is 4508 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb6c1ff490 con 0x7feb6c0585e0 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1431119565 0 0) 0x7f2540003610 con 0x7f2570157780 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f257013c0a0 con 0x7f2570157780 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2564003100 con 0x7f2570157780 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f2547fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f2547fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f2547fff640 10 monclient(hunting): dump: 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.413+0000 7f2547fff640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7f4d37fff640 1 -- 10.20.193.90:0/2747018121 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3992672206 0 0) 0x7f4d48163ed0 con 0x7f4d48159e40 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7f4d37fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7f4d37fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7f4d37fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7f4d37fff640 10 monclient(hunting): my global_id is 4511 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7f4d37fff640 1 -- 10.20.193.90:0/2747018121 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d30003610 con 0x7f4d48159e40 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7fafef7fe640 1 -- 10.20.193.90:0/3518157365 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7faff4003380 con 0x7faff80585e0 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7fafef7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7fafef7fe640 1 -- 10.20.193.90:0/3518157365 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7faff40037c0 con 0x7faff80585e0 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7fafef7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7fafef7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7fafef7fe640 10 monclient: dump: 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7faffee7b640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7faffee7b640 4 set_mon_vals no callback set 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7f788bfff640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f78940ba350 0x7f7894057ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54402/0 (socket says 10.20.193.90:54402) 2026-02-25T15:01:42.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7f788bfff640 1 -- 10.20.193.90:0/1371054926 learned_addr learned my addr 10.20.193.90:0/1371054926 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7faffee7b640 10 monclient: shutdown 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7faffee7b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.414+0000 7faffee7b640 1 -- 10.20.193.90:0/3518157365 >> v1:10.20.193.93:6789/0 conn(0x7faff80585e0 legacy=0x7faff8166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7faffee7b640 1 -- 10.20.193.90:0/3518157365 shutdown_connections 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7faffee7b640 1 -- 10.20.193.90:0/3518157365 >> 10.20.193.90:0/3518157365 conn(0x7faff80b7990 msgr2=0x7faff80b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7faffee7b640 1 -- 10.20.193.90:0/3518157365 shutdown_connections 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7faffee7b640 1 -- 10.20.193.90:0/3518157365 wait complete. 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7faffee7b640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7faffee7b640 10 monclient: monmap: 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7faffee7b640 1 librados: starting msgr at 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7faffee7b640 1 librados: starting objecter 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 172120711 0 0) 0x7feb3c003610 con 0x7feb6c0b9f80 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7feb6c1ff770 con 0x7feb6c0b9f80 2026-02-25T15:01:42.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7feb54003120 con 0x7feb6c0b9f80 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7feb617fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7feb617fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7feb617fa640 10 monclient(hunting): dump: 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7feb617fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7faffee7b640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.415+0000 7faffee7b640 1 Processor -- start 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7faffee7b640 1 -- start start 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7faffee7b640 1 librados: setting wanted keys 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7faffee7b640 1 librados: calling monclient init 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7faffee7b640 10 monclient: init 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7f0def577640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f0df00b5da0 0x7f0df00b7e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46684/0 (socket says 10.20.193.90:46684) 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7f0def577640 1 -- 10.20.193.90:0/314554311 learned_addr learned my addr 10.20.193.90:0/314554311 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7faffee7b640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7faffee7b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7faffee7b640 10 monclient(hunting): picked mon.noname-a con 0x7faff80585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7faffee7b640 10 monclient(hunting): picked mon.noname-b con 0x7faff80b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7faffee7b640 10 monclient(hunting): picked mon.noname-c con 0x7faff80ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7faffee7b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff813a0a0 con 0x7faff80585e0 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7faffee7b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff8139240 con 0x7faff80b9f80 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7faffee7b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff8139520 con 0x7faff80ba350 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7faffee7b640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.416+0000 7faffee7b640 10 monclient(hunting): authenticate will time out at 2992.016709s 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3252465302 0 0) 0x7f257013c0a0 con 0x7f2570157780 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 >> v1:10.20.193.93:6789/0 conn(0x7f2570157b50 legacy=0x7f257013b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 >> v1:10.20.193.90:6789/0 conn(0x7f2570164d00 legacy=0x7f25701211c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f2547fff640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f2547fff640 1 monclient: found mon.c 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f2547fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f2547fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25701206e0 con 0x7f2570157780 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f2547fff640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f2547fff640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f2547fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f257584d640 5 monclient: authenticate success, global_id 4505 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f257584d640 10 monclient: _renew_subs 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f257584d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f257584d640 1 -- 10.20.193.90:0/359940460 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f25701209c0 con 0x7f2570157780 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f257584d640 10 client.4505.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f257584d640 10 monclient: _renew_subs 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f257584d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f257584d640 1 -- 10.20.193.90:0/359940460 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f25701221c0 con 0x7f2570157780 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.417+0000 7f257584d640 1 librados: init done 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.418+0000 7f257584d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.418+0000 7f257584d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.418+0000 7f257584d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.418+0000 7f257584d640 1 -- 10.20.193.90:0/359940460 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f257015ea20 con 0x7f2570157780 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7fcc99573640 1 -- 10.20.193.90:0/1911448125 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1788130643 0 0) 0x7fcc8c002c90 con 0x7fcc9c0585e0 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7fcc99573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7fcc99573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7fcc99573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7fcc99573640 10 monclient(hunting): my global_id is 4500 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d37fff640 1 -- 10.20.193.90:0/2747018121 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2514013952 0 0) 0x7f4d30003610 con 0x7f4d48159e40 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7fcc99573640 1 -- 10.20.193.90:0/1911448125 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc88003610 con 0x7fcc9c0585e0 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d37fff640 1 -- 10.20.193.90:0/2747018121 >> v1:10.20.193.93:6789/0 conn(0x7f4d48159a70 legacy=0x7f4d480b6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d37fff640 1 -- 10.20.193.90:0/2747018121 >> v1:10.20.193.90:6789/0 conn(0x7f4d48058070 legacy=0x7f4d48058460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d37fff640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d37fff640 1 monclient: found mon.noname-c 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d37fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d37fff640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d37fff640 1 -- 10.20.193.90:0/2747018121 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d48057590 con 0x7f4d48159e40 2026-02-25T15:01:42.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d37fff640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d37fff640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d37fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d37fff640 1 -- 10.20.193.90:0/2747018121 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4d3c002df0 con 0x7f4d48159e40 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d37fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d37fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d37fff640 10 monclient: dump: 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.420+0000 7f4d4f71a640 5 monclient: authenticate success, global_id 4511 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.421+0000 7f4d4f71a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.421+0000 7f73de682640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.421+0000 7f73de682640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.421+0000 7f73de682640 10 monclient: monmap: 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.422115+0000 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.422115+0000 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.421+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1734134870 0 0) 0x7feb6c166030 con 0x7feb6c0ba350 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.421+0000 7feb617fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.421+0000 7feb617fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.421+0000 7feb617fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.421+0000 7feb617fa640 10 monclient(hunting): my global_id is 4494 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.421+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb3c003610 con 0x7feb6c0ba350 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.421+0000 7f73de682640 1 Processor -- start 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.421+0000 7f73de682640 1 -- start start 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.421+0000 7f73de682640 10 monclient: init 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.422+0000 7f73de682640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.422+0000 7f73de682640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.422+0000 7f73de682640 10 monclient(hunting): picked mon.noname-a con 0x7f73d80b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.422+0000 7f73de682640 10 monclient(hunting): picked mon.noname-b con 0x7f73d80ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.422+0000 7f73de682640 10 monclient(hunting): picked mon.noname-c con 0x7f73d80585e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.422+0000 7f73de682640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d8163930 con 0x7f73d80b9f80 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.422+0000 7f73de682640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d8163bb0 con 0x7f73d80ba350 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.422+0000 7f73de682640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d8163e90 con 0x7f73d80585e0 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.422+0000 7f73de682640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.422+0000 7f73de682640 10 monclient(hunting): authenticate will time out at 2992.022544s 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.424+0000 7f2ca80aa640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.424+0000 7f2ca80aa640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.424+0000 7f2ca80aa640 10 monclient: monmap: 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.425073+0000 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.425073+0000 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.442 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.424+0000 7f2ca80aa640 1 Processor -- start 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.424+0000 7f9129ffb640 1 -- 10.20.193.90:0/3114250905 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f911c003340 con 0x7f912c162f10 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f9129ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f9129ffb640 1 -- 10.20.193.90:0/3114250905 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f911c003780 con 0x7f912c162f10 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f9129ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f9129ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f9129ffb640 10 monclient: dump: 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.424+0000 7f2ca80aa640 1 -- start start 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f9131999640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f9131999640 4 set_mon_vals no callback set 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f2ca80aa640 10 monclient: init 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f0dedd74640 1 -- 10.20.193.90:0/314554311 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1982628937 0 0) 0x7f0df00b38f0 con 0x7f0df00b5da0 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f0dedd74640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f0dedd74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f0dedd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f0dedd74640 10 monclient(hunting): my global_id is 4522 2026-02-25T15:01:42.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f0dedd74640 1 -- 10.20.193.90:0/314554311 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ddc003610 con 0x7f0df00b5da0 2026-02-25T15:01:42.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f9131999640 10 monclient: shutdown 2026-02-25T15:01:42.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f9131999640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f9131999640 1 -- 10.20.193.90:0/3114250905 >> v1:10.20.193.93:6789/0 conn(0x7f912c162f10 legacy=0x7f912c0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f2ca80aa640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f2ca80aa640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:42.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f2ca80aa640 10 monclient(hunting): picked mon.noname-b con 0x7f2ca00b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f2ca80aa640 10 monclient(hunting): picked mon.noname-a con 0x7f2ca00ba350 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f2ca80aa640 10 monclient(hunting): picked mon.noname-c con 0x7f2ca00585e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f9131999640 1 -- 10.20.193.90:0/3114250905 shutdown_connections 2026-02-25T15:01:42.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f2ca80aa640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca0163930 con 0x7f2ca00ba350 2026-02-25T15:01:42.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f9131999640 1 -- 10.20.193.90:0/3114250905 >> 10.20.193.90:0/3114250905 conn(0x7f912c0b2ef0 msgr2=0x7f912c0b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f2ca80aa640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca0163bb0 con 0x7f2ca00b9f80 2026-02-25T15:01:42.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f2ca80aa640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca0163e90 con 0x7f2ca00585e0 2026-02-25T15:01:42.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f9131999640 1 -- 10.20.193.90:0/3114250905 shutdown_connections 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f2ca80aa640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f2ca80aa640 10 monclient(hunting): authenticate will time out at 2992.025876s 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.425+0000 7f9131999640 1 -- 10.20.193.90:0/3114250905 wait complete. 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.426+0000 7f460b7fe640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f460c0585e0 0x7f460c166cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46696/0 (socket says 10.20.193.90:46696) 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.426+0000 7f460b7fe640 1 -- 10.20.193.90:0/1282412004 learned_addr learned my addr 10.20.193.90:0/1282412004 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.426+0000 7f9131999640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.426+0000 7f9131999640 10 monclient: monmap: 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.426+0000 7f9131999640 1 librados: starting msgr at 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.426+0000 7f9131999640 1 librados: starting objecter 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.426+0000 7f9131999640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.426+0000 7f9131999640 1 Processor -- start 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.426+0000 7f9131999640 1 -- start start 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.426+0000 7f9131999640 1 librados: setting wanted keys 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.426+0000 7f9131999640 1 librados: calling monclient init 2026-02-25T15:01:42.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.426+0000 7f9131999640 10 monclient: init 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3924398704 0 0) 0x7feb6c1ff770 con 0x7feb6c0b9f80 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 >> v1:10.20.193.94:6789/0 conn(0x7feb6c0585e0 legacy=0x7feb6c1febf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 >> v1:10.20.193.90:6789/0 conn(0x7feb6c0ba350 legacy=0x7feb6c153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb617fa640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb617fa640 1 monclient: found mon.b 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb617fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb617fa640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c1ffa50 con 0x7feb6c0b9f80 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb617fa640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb617fa640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb617fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb72c8b640 5 monclient: authenticate success, global_id 4513 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb72c8b640 10 monclient: _renew_subs 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb72c8b640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb72c8b640 1 -- 10.20.193.90:0/3231812946 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c1fb790 con 0x7feb6c0b9f80 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb72c8b640 10 client.4513.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb72c8b640 10 monclient: _renew_subs 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb72c8b640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb72c8b640 1 -- 10.20.193.90:0/3231812946 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7feb6c1fc270 con 0x7feb6c0b9f80 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb72c8b640 1 librados: init done 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f9131999640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f9131999640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:42.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f9131999640 10 monclient(hunting): picked mon.noname-b con 0x7f912c056c70 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1807458193 0 0) 0x7fb4200a83a0 con 0x7fb420164d00 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f9131999640 10 monclient(hunting): picked mon.noname-c con 0x7f912c155b20 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f9131999640 10 monclient(hunting): picked mon.noname-a con 0x7f912c162f10 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f9131999640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f912c0a75b0 con 0x7f912c162f10 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f9131999640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f912c0abba0 con 0x7f912c056c70 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb4200a80c0 con 0x7fb420164d00 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f9131999640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f912c0abe80 con 0x7f912c155b20 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f9131999640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f9131999640 10 monclient(hunting): authenticate will time out at 2992.027472s 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb4100036d0 con 0x7fb420164d00 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7fb3ff7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7fb3ff7fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7fb3ff7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.447 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f0dedd74640 1 -- 10.20.193.90:0/314554311 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3034976278 0 0) 0x7f0df00b3610 con 0x7f0df00b3190 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f0dedd74640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f0dedd74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f0dedd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f0dedd74640 10 monclient(hunting): my global_id is 4503 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7f0dedd74640 1 -- 10.20.193.90:0/314554311 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0df00b38f0 con 0x7f0df00b3190 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb72c8b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb72c8b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb72c8b640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7feb72c8b640 1 -- 10.20.193.90:0/3231812946 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c15c600 con 0x7feb6c0b9f80 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1043999392 0 0) 0x7fb4200af7b0 con 0x7fb420157780 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 >> v1:10.20.193.94:6789/0 conn(0x7fb420157b50 legacy=0x7fb4200af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 >> v1:10.20.193.90:6789/0 conn(0x7fb420164d00 legacy=0x7fb4200b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7fb3ff7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7fb3ff7fe640 1 monclient: found mon.b 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7fb3ff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7fb3ff7fe640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.427+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4200a8680 con 0x7fb420157780 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb3ff7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb3ff7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb3ff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb425082640 5 monclient: authenticate success, global_id 4501 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb425082640 10 monclient: _renew_subs 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb425082640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb425082640 1 -- 10.20.193.90:0/2259846989 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb4200a8960 con 0x7fb420157780 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb425082640 10 client.4501.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb425082640 10 monclient: _renew_subs 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb425082640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb425082640 1 -- 10.20.193.90:0/2259846989 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb4200a8f20 con 0x7fb420157780 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb425082640 1 librados: init done 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7f788b7fe640 1 -- 10.20.193.90:0/1371054926 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1959224828 0 0) 0x7f7890002c90 con 0x7f78940585e0 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7f788b7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7f788b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7f788b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7f788b7fe640 10 monclient(hunting): my global_id is 4506 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7f788b7fe640 1 -- 10.20.193.90:0/1371054926 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7880003610 con 0x7f78940585e0 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb425082640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb425082640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb425082640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7fb425082640 1 -- 10.20.193.90:0/2259846989 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb42015ea20 con 0x7fb420157780 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7f73d7fff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f73d80b9f80 0x7f73d8163220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45174/0 (socket says 10.20.193.90:45174) 2026-02-25T15:01:42.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.428+0000 7f73d7fff640 1 -- 10.20.193.90:0/660775841 learned_addr learned my addr 10.20.193.90:0/660775841 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.430+0000 7faffcbf0640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7faff80585e0 0x7faff81624a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45172/0 (socket says 10.20.193.90:45172) 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.430+0000 7faffcbf0640 1 -- 10.20.193.90:0/4251103413 learned_addr learned my addr 10.20.193.90:0/4251103413 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f73d6ffd640 1 -- 10.20.193.90:0/660775841 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1685633454 0 0) 0x7f73c4002910 con 0x7f73d80b9f80 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f73d6ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f73d6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f73d6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f73d6ffd640 10 monclient(hunting): my global_id is 4509 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f73d6ffd640 1 -- 10.20.193.90:0/660775841 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73bc003610 con 0x7f73d80b9f80 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0dedd74640 1 -- 10.20.193.90:0/314554311 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 321417585 0 0) 0x7f0df00b38f0 con 0x7f0df00b3190 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0dedd74640 1 -- 10.20.193.90:0/314554311 >> v1:10.20.193.94:6789/0 conn(0x7f0df015b0a0 legacy=0x7f0df0162300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0dedd74640 1 -- 10.20.193.90:0/314554311 >> v1:10.20.193.93:6789/0 conn(0x7f0df00b5da0 legacy=0x7f0df00b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0dedd74640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0dedd74640 1 monclient: found mon.noname-a 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0dedd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0dedd74640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0dedd74640 1 -- 10.20.193.90:0/314554311 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df00b3eb0 con 0x7f0df00b3190 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0dedd74640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0dedd74640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0dedd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0dedd74640 1 -- 10.20.193.90:0/314554311 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0dd8002dd0 con 0x7f0df00b3190 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0dedd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0dedd74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0dedd74640 10 monclient: dump: 2026-02-25T15:01:42.450 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0df5171640 5 monclient: authenticate success, global_id 4503 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f0df5171640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f788b7fe640 1 -- 10.20.193.90:0/1371054926 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1932789123 0 0) 0x7f7880003610 con 0x7f78940585e0 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f788b7fe640 1 -- 10.20.193.90:0/1371054926 >> v1:10.20.193.94:6789/0 conn(0x7f78940ba350 legacy=0x7f7894057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f788b7fe640 1 -- 10.20.193.90:0/1371054926 >> v1:10.20.193.93:6789/0 conn(0x7f78940b9f80 legacy=0x7f7894163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f788b7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f788b7fe640 1 monclient: found mon.noname-a 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f788b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f788b7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f788b7fe640 1 -- 10.20.193.90:0/1371054926 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78940573f0 con 0x7f78940585e0 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f788b7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f788b7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f788b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f789b18a640 5 monclient: authenticate success, global_id 4506 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.435+0000 7f789b18a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.436+0000 7fafed7fa640 10 client.?.objecter ms_handle_connect 0x7faff80585e0 2026-02-25T15:01:42.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.436+0000 7fafed7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.436+0000 7f2ca561e640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f2ca00ba350 0x7f2ca0057ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45180/0 (socket says 10.20.193.90:45180) 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.436+0000 7f2ca561e640 1 -- 10.20.193.90:0/1287177133 learned_addr learned my addr 10.20.193.90:0/1287177133 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcc99573640 1 -- 10.20.193.90:0/1911448125 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1132823994 0 0) 0x7fcc88003610 con 0x7fcc9c0585e0 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcc99573640 1 -- 10.20.193.90:0/1911448125 >> v1:10.20.193.94:6789/0 conn(0x7fcc9c0b9f80 legacy=0x7fcc9c163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcc99573640 1 -- 10.20.193.90:0/1911448125 >> v1:10.20.193.93:6789/0 conn(0x7fcc9c0ba350 legacy=0x7fcc9c057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcc99573640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcc99573640 1 monclient: found mon.noname-a 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcc99573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcc99573640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcc99573640 1 -- 10.20.193.90:0/1911448125 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9c0573f0 con 0x7fcc9c0585e0 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcc99573640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcc99573640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcc99573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcc99573640 1 -- 10.20.193.90:0/1911448125 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fcc8c003340 con 0x7fcc9c0585e0 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcc99573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcc99573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcc99573640 10 monclient: dump: 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcca0ee8640 5 monclient: authenticate success, global_id 4500 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7fcca0ee8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7f912b7fe640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f912c162f10 0x7f912c1f7fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45186/0 (socket says 10.20.193.90:45186) 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.437+0000 7f912b7fe640 1 -- 10.20.193.90:0/433540121 learned_addr learned my addr 10.20.193.90:0/433540121 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.438+0000 7f9103fff640 10 client.?.objecter ms_handle_connect 0x7f912c162f10 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.438+0000 7f9103fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f4609ffb640 1 -- 10.20.193.90:0/1282412004 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 440919254 0 0) 0x7f45f4002910 con 0x7f460c0ba350 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f4609ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f4609ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f4609ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f4609ffb640 10 monclient(hunting): my global_id is 4512 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f4609ffb640 1 -- 10.20.193.90:0/1282412004 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45f8003610 con 0x7f460c0ba350 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3517628846 0 0) 0x7faff813a0a0 con 0x7faff80585e0 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7fafed7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7fafed7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7fafed7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7fafed7fa640 10 monclient(hunting): my global_id is 4515 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafc8003610 con 0x7faff80585e0 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73d6ffd640 1 -- 10.20.193.90:0/660775841 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 359465824 0 0) 0x7f73bc003610 con 0x7f73d80b9f80 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73d6ffd640 1 -- 10.20.193.90:0/660775841 >> v1:10.20.193.94:6789/0 conn(0x7f73d80585e0 legacy=0x7f73d8166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73d6ffd640 1 -- 10.20.193.90:0/660775841 >> v1:10.20.193.93:6789/0 conn(0x7f73d80ba350 legacy=0x7f73d8057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73d6ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73d6ffd640 1 monclient: found mon.noname-a 2026-02-25T15:01:42.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73d6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73d6ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73d6ffd640 1 -- 10.20.193.90:0/660775841 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d80573f0 con 0x7f73d80b9f80 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73d6ffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73d6ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73d6ffd640 1 -- 10.20.193.90:0/660775841 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f73c4002fc0 con 0x7f73d80b9f80 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73d6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73d6ffd640 10 monclient: dump: 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73de682640 5 monclient: authenticate success, global_id 4509 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f73de682640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7fcc99573640 1 -- 10.20.193.90:0/1911448125 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fcc8c003550 con 0x7fcc9c0585e0 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7fcc99573640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7fcc99573640 1 -- 10.20.193.90:0/1911448125 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fcc8c004d10 con 0x7fcc9c0585e0 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7fcc99573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7fcc99573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7fcc99573640 10 monclient: dump: 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.454 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f2ca4e1d640 1 -- 10.20.193.90:0/1287177133 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 6994751 0 0) 0x7f2c94002910 con 0x7f2ca00ba350 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f2ca4e1d640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f2ca4e1d640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f2ca4e1d640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f2ca4e1d640 10 monclient(hunting): my global_id is 4518 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7f2ca4e1d640 1 -- 10.20.193.90:0/1287177133 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c88003610 con 0x7f2ca00ba350 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7fcca0ee8640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.443+0000 7fcca0ee8640 4 set_mon_vals no callback set 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7fcca0ee8640 10 monclient: shutdown 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7fcca0ee8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7fcca0ee8640 1 -- 10.20.193.90:0/1911448125 >> v1:10.20.193.90:6789/0 conn(0x7fcc9c0585e0 legacy=0x7fcc9c166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f788b7fe640 1 -- 10.20.193.90:0/1371054926 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7890003340 con 0x7f78940585e0 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f788b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7fcca0ee8640 1 -- 10.20.193.90:0/1911448125 shutdown_connections 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f788b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f788b7fe640 10 monclient: dump: 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.455 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7fcca0ee8640 1 -- 10.20.193.90:0/1911448125 >> 10.20.193.90:0/1911448125 conn(0x7fcc9c0b7990 msgr2=0x7fcc9c0b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f788b7fe640 1 -- 10.20.193.90:0/1371054926 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f78900035d0 con 0x7f78940585e0 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f789b18a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f788b7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f788b7fe640 1 -- 10.20.193.90:0/1371054926 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7890003a80 con 0x7f78940585e0 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f788b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f788b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f788b7fe640 10 monclient: dump: 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f789b18a640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f789b18a640 4 set_mon_vals no callback set 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f0dedd74640 1 -- 10.20.193.90:0/314554311 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0dd8002fe0 con 0x7f0df00b3190 2026-02-25T15:01:42.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f0dedd74640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f0dedd74640 1 -- 10.20.193.90:0/314554311 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0dd8003420 con 0x7f0df00b3190 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f0dedd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f0dedd74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f0dedd74640 10 monclient: dump: 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f0df5171640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f0df5171640 4 set_mon_vals no callback set 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7fcca0ee8640 1 -- 10.20.193.90:0/1911448125 shutdown_connections 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7fcca0ee8640 1 -- 10.20.193.90:0/1911448125 wait complete. 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f789b18a640 10 monclient: shutdown 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f789b18a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f789b18a640 1 -- 10.20.193.90:0/1371054926 >> v1:10.20.193.90:6789/0 conn(0x7f78940585e0 legacy=0x7f7894166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f0df5171640 10 monclient: shutdown 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f0df5171640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f0df5171640 1 -- 10.20.193.90:0/314554311 >> v1:10.20.193.90:6789/0 conn(0x7f0df00b3190 legacy=0x7f0df00b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f789b18a640 1 -- 10.20.193.90:0/1371054926 shutdown_connections 2026-02-25T15:01:42.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f789b18a640 1 -- 10.20.193.90:0/1371054926 >> 10.20.193.90:0/1371054926 conn(0x7f78940b7990 msgr2=0x7f78940b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f0df5171640 1 -- 10.20.193.90:0/314554311 shutdown_connections 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f0df5171640 1 -- 10.20.193.90:0/314554311 >> 10.20.193.90:0/314554311 conn(0x7f0df00b5290 msgr2=0x7f0df00b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f789b18a640 1 -- 10.20.193.90:0/1371054926 shutdown_connections 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7fcca0ee8640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f789b18a640 1 -- 10.20.193.90:0/1371054926 wait complete. 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f0df5171640 1 -- 10.20.193.90:0/314554311 shutdown_connections 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7fcca0ee8640 10 monclient: monmap: 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.444+0000 7f0df5171640 1 -- 10.20.193.90:0/314554311 wait complete. 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 1 librados: starting msgr at 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 1 librados: starting objecter 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 1 Processor -- start 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 1 -- start start 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 1 librados: setting wanted keys 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 1 librados: calling monclient init 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 10 monclient: init 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f0df5171640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f0df5171640 10 monclient: monmap: 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f789b18a640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f789b18a640 10 monclient: monmap: 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f789b18a640 1 librados: starting msgr at 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f789b18a640 1 librados: starting objecter 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f789b18a640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f789b18a640 1 Processor -- start 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 10 monclient(hunting): picked mon.noname-c con 0x7fcc9c0585e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 10 monclient(hunting): picked mon.noname-a con 0x7fcc9c0b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 10 monclient(hunting): picked mon.noname-b con 0x7fcc9c0ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc9c1608f0 con 0x7fcc9c0b9f80 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc9c160330 con 0x7fcc9c0ba350 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc9c160610 con 0x7fcc9c0585e0 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7fcca0ee8640 10 monclient(hunting): authenticate will time out at 2992.045773s 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f789b18a640 1 -- start start 2026-02-25T15:01:42.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f789b18a640 1 librados: setting wanted keys 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f789b18a640 1 librados: calling monclient init 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f0df5171640 1 librados: starting msgr at 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f0df5171640 1 librados: starting objecter 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f789b18a640 10 monclient: init 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f0df5171640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f0df5171640 1 Processor -- start 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f4609ffb640 1 -- 10.20.193.90:0/1282412004 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1629970375 0 0) 0x7f45f8003610 con 0x7f460c0ba350 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f4609ffb640 1 -- 10.20.193.90:0/1282412004 >> v1:10.20.193.94:6789/0 conn(0x7f460c0b9f80 legacy=0x7f460c163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f4609ffb640 1 -- 10.20.193.90:0/1282412004 >> v1:10.20.193.93:6789/0 conn(0x7f460c0585e0 legacy=0x7f460c166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f4609ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f4609ffb640 1 monclient: found mon.noname-a 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f4609ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f4609ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f4609ffb640 1 -- 10.20.193.90:0/1282412004 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460c0573f0 con 0x7f460c0ba350 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f4609ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f4609ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f4609ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f0df5171640 1 -- start start 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f0df5171640 1 librados: setting wanted keys 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f0df5171640 1 librados: calling monclient init 2026-02-25T15:01:42.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f0df5171640 10 monclient: init 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f461167c640 5 monclient: authenticate success, global_id 4512 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f461167c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f4609ffb640 1 -- 10.20.193.90:0/1282412004 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f45f4002fc0 con 0x7f460c0ba350 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.445+0000 7f4609ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f4609ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f4609ffb640 10 monclient: dump: 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f461167c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f9103fff640 1 -- 10.20.193.90:0/433540121 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 906320362 0 0) 0x7f912c0a75b0 con 0x7f912c162f10 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f9103fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f9103fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f9103fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f9103fff640 10 monclient(hunting): my global_id is 4521 2026-02-25T15:01:42.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f9103fff640 1 -- 10.20.193.90:0/433540121 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90fc003610 con 0x7f912c162f10 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f789b18a640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f789b18a640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f789b18a640 10 monclient(hunting): picked mon.noname-c con 0x7f78940585e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f789b18a640 10 monclient(hunting): picked mon.noname-a con 0x7f78940b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f789b18a640 10 monclient(hunting): picked mon.noname-b con 0x7f78940ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f789b18a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f789413a0a0 con 0x7f78940b9f80 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f789b18a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7894139240 con 0x7f78940ba350 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f789b18a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7894139520 con 0x7f78940585e0 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f789b18a640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f789b18a640 10 monclient(hunting): authenticate will time out at 2992.046375s 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f0df5171640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f0df5171640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f0df5171640 10 monclient(hunting): picked mon.noname-c con 0x7f0df00b5da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f0df5171640 10 monclient(hunting): picked mon.noname-b con 0x7f0df015b0a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f0df5171640 10 monclient(hunting): picked mon.noname-a con 0x7f0df01fd410 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f0df5171640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df00b2190 con 0x7f0df01fd410 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f0df5171640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df01604e0 con 0x7f0df015b0a0 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f0df5171640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df00b06a0 con 0x7f0df00b5da0 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f0df5171640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.446+0000 7f0df5171640 10 monclient(hunting): authenticate will time out at 2992.046567s 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.453+0000 7f73d6ffd640 1 -- 10.20.193.90:0/660775841 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f73c40031d0 con 0x7f73d80b9f80 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.453+0000 7f73d6ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.453+0000 7f73d6ffd640 1 -- 10.20.193.90:0/660775841 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f73c4004990 con 0x7f73d80b9f80 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.453+0000 7f73d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.453+0000 7f73d6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.453+0000 7f73d6ffd640 10 monclient: dump: 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.453+0000 7f73de682640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.453+0000 7f73de682640 4 set_mon_vals no callback set 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.453+0000 7f73de682640 10 monclient: shutdown 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.453+0000 7f73de682640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.453+0000 7f73de682640 1 -- 10.20.193.90:0/660775841 >> v1:10.20.193.90:6789/0 conn(0x7f73d80b9f80 legacy=0x7f73d8163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.453+0000 7f73de682640 1 -- 10.20.193.90:0/660775841 shutdown_connections 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.453+0000 7f73de682640 1 -- 10.20.193.90:0/660775841 >> 10.20.193.90:0/660775841 conn(0x7f73d80b7990 msgr2=0x7f73d80b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.453+0000 7f73de682640 1 -- 10.20.193.90:0/660775841 shutdown_connections 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.453+0000 7f73de682640 1 -- 10.20.193.90:0/660775841 wait complete. 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f73de682640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f73de682640 10 monclient: monmap: 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f73de682640 1 librados: starting msgr at 2026-02-25T15:01:42.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f73de682640 1 librados: starting objecter 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca4e1d640 1 -- 10.20.193.90:0/1287177133 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2913226144 0 0) 0x7f2c88003610 con 0x7f2ca00ba350 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca4e1d640 1 -- 10.20.193.90:0/1287177133 >> v1:10.20.193.94:6789/0 conn(0x7f2ca00585e0 legacy=0x7f2ca0166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca4e1d640 1 -- 10.20.193.90:0/1287177133 >> v1:10.20.193.93:6789/0 conn(0x7f2ca00b9f80 legacy=0x7f2ca0163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f73de682640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f73de682640 1 Processor -- start 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca4e1d640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca4e1d640 1 monclient: found mon.noname-a 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca4e1d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca4e1d640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca4e1d640 1 -- 10.20.193.90:0/1287177133 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca00573f0 con 0x7f2ca00ba350 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca4e1d640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca4e1d640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca4e1d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca4e1d640 1 -- 10.20.193.90:0/1287177133 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2c94002fc0 con 0x7f2ca00ba350 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca4e1d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca4e1d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca4e1d640 10 monclient: dump: 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.465 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca80aa640 5 monclient: authenticate success, global_id 4518 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f2ca80aa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f73de682640 1 -- start start 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f73de682640 1 librados: setting wanted keys 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f73de682640 1 librados: calling monclient init 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.454+0000 7f73de682640 10 monclient: init 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f73de682640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f73de682640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f9103fff640 1 -- 10.20.193.90:0/433540121 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2915769550 0 0) 0x7f90fc003610 con 0x7f912c162f10 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f73de682640 10 monclient(hunting): picked mon.noname-b con 0x7f73d80585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f73de682640 10 monclient(hunting): picked mon.noname-c con 0x7f73d80b9f80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f73de682640 10 monclient(hunting): picked mon.noname-a con 0x7f73d80ba350 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f73de682640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d8155880 con 0x7f73d80ba350 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f73de682640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d81fd090 con 0x7f73d80585e0 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f73de682640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d81fd370 con 0x7f73d80b9f80 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f73de682640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f73de682640 10 monclient(hunting): authenticate will time out at 2992.055324s 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f9103fff640 1 -- 10.20.193.90:0/433540121 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f912c0a75b0 con 0x7f912c162f10 2026-02-25T15:01:42.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f9103fff640 1 -- 10.20.193.90:0/433540121 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f911c003690 con 0x7f912c162f10 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f9103fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f9103fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f9103fff640 10 monclient(hunting): dump: 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f9103fff640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f592a5a9640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f592a5a9640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f592a5a9640 10 monclient: monmap: 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.456221+0000 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.456221+0000 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7fcc99d74640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fcc9c0b9f80 0x7fcc9c15ae60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45194/0 (socket says 10.20.193.90:45194) 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7fcc99d74640 1 -- 10.20.193.90:0/129879591 learned_addr learned my addr 10.20.193.90:0/129879591 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7fe52125b640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7fe52125b640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7fe52125b640 10 monclient: monmap: 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.456485+0000 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.456485+0000 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f60a8dc8640 10 monclient: get_monmap_and_config 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f60a8dc8640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.455+0000 7f60a8dc8640 10 monclient: monmap: 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:42.456506+0000 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:42.456506+0000 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f592a5a9640 1 Processor -- start 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f592a5a9640 1 -- start start 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f592a5a9640 10 monclient: init 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7fe52125b640 1 Processor -- start 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7fe52125b640 1 -- start start 2026-02-25T15:01:42.468 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7fe52125b640 10 monclient: init 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f60a8dc8640 1 Processor -- start 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f592a5a9640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f592a5a9640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f60a8dc8640 1 -- start start 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f60a8dc8640 10 monclient: init 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f592a5a9640 10 monclient(hunting): picked mon.noname-a con 0x7f59240b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f592a5a9640 10 monclient(hunting): picked mon.noname-c con 0x7f59240ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f592a5a9640 10 monclient(hunting): picked mon.noname-b con 0x7f59240585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f592a5a9640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5924163930 con 0x7f59240b9f80 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f592a5a9640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5924163bb0 con 0x7f59240585e0 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f592a5a9640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5924163e90 con 0x7f59240ba350 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f592a5a9640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7f592a5a9640 10 monclient(hunting): authenticate will time out at 2992.056717s 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7fe52125b640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7fe52125b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7fe52125b640 10 monclient(hunting): picked mon.noname-a con 0x7fe51c0b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7fe52125b640 10 monclient(hunting): picked mon.noname-b con 0x7fe51c0ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.469 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7fe52125b640 10 monclient(hunting): picked mon.noname-c con 0x7fe51c0585e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7fe52125b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe51c163930 con 0x7fe51c0b9f80 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7fe52125b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe51c163bb0 con 0x7fe51c0ba350 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7fe52125b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe51c163e90 con 0x7fe51c0585e0 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7fe52125b640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7fe52125b640 10 monclient(hunting): authenticate will time out at 2992.056905s 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.456+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1867247856 0 0) 0x7fafc8003610 con 0x7faff80585e0 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faff813a0a0 con 0x7faff80585e0 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fafe8003200 con 0x7faff80585e0 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7fafed7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7fafed7fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7fafed7fa640 10 monclient(hunting): dump: 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7fafed7fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7f60a8dc8640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7f60a8dc8640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7f60a8dc8640 10 monclient(hunting): picked mon.noname-a con 0x7f60a40b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7f60a8dc8640 10 monclient(hunting): picked mon.noname-c con 0x7f60a40ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7f60a8dc8640 10 monclient(hunting): picked mon.noname-b con 0x7f60a40585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7f60a8dc8640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a4163930 con 0x7f60a40b9f80 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7f60a8dc8640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a4163bb0 con 0x7f60a40585e0 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7f60a8dc8640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a4163e90 con 0x7f60a40ba350 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7f60a8dc8640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7f60a8dc8640 10 monclient(hunting): authenticate will time out at 2992.057371s 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7fcc82ffd640 10 client.?.objecter ms_handle_connect 0x7fcc9c0b9f80 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7fcc82ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7f0def577640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f0df01fd410 0x7f0df0154150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45204/0 (socket says 10.20.193.90:45204) 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.457+0000 7f0def577640 1 -- 10.20.193.90:0/3673364815 learned_addr learned my addr 10.20.193.90:0/3673364815 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.458+0000 7f0dcf7fe640 10 client.?.objecter ms_handle_connect 0x7f0df01fd410 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.458+0000 7f0dcf7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.461+0000 7f788bfff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f78940b9f80 0x7f78941656b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45202/0 (socket says 10.20.193.90:45202) 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.461+0000 7f788bfff640 1 -- 10.20.193.90:0/3904297474 learned_addr learned my addr 10.20.193.90:0/3904297474 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 443460947 0 0) 0x7fcc9c1608f0 con 0x7fcc9c0b9f80 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7fcc82ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7fcc82ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7fcc82ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7fcc82ffd640 10 monclient(hunting): my global_id is 4524 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f4609ffb640 1 -- 10.20.193.90:0/1282412004 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f45f40031d0 con 0x7f460c0ba350 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f4609ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f461167c640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f4609ffb640 1 -- 10.20.193.90:0/1282412004 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f45f4003700 con 0x7f460c0ba350 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f461167c640 4 set_mon_vals no callback set 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f4609ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f4609ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f4609ffb640 10 monclient: dump: 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc68003610 con 0x7fcc9c0b9f80 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f78897fa640 10 client.?.objecter ms_handle_connect 0x7f78940b9f80 2026-02-25T15:01:42.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f78897fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f461167c640 10 monclient: shutdown 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f461167c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f461167c640 1 -- 10.20.193.90:0/1282412004 >> v1:10.20.193.90:6789/0 conn(0x7f460c0ba350 legacy=0x7f460c057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f461167c640 1 -- 10.20.193.90:0/1282412004 shutdown_connections 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f461167c640 1 -- 10.20.193.90:0/1282412004 >> 10.20.193.90:0/1282412004 conn(0x7f460c0b7990 msgr2=0x7f460c0b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f461167c640 1 -- 10.20.193.90:0/1282412004 shutdown_connections 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.462+0000 7f461167c640 1 -- 10.20.193.90:0/1282412004 wait complete. 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.463+0000 7f461167c640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.463+0000 7f461167c640 10 monclient: monmap: 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.463+0000 7f461167c640 1 librados: starting msgr at 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.463+0000 7f461167c640 1 librados: starting objecter 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.463+0000 7f461167c640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.463+0000 7f461167c640 1 Processor -- start 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.463+0000 7f461167c640 1 -- start start 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.463+0000 7f461167c640 1 librados: setting wanted keys 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.463+0000 7f461167c640 1 librados: calling monclient init 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.463+0000 7f461167c640 10 monclient: init 2026-02-25T15:01:42.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.463+0000 7f461167c640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.463+0000 7f461167c640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f461167c640 10 monclient(hunting): picked mon.noname-c con 0x7f460c0585e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f461167c640 10 monclient(hunting): picked mon.noname-b con 0x7f460c0b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f461167c640 10 monclient(hunting): picked mon.noname-a con 0x7f460c0ba350 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f461167c640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f460c1fed70 con 0x7f460c0ba350 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f461167c640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f460c165c30 con 0x7f460c0b9f80 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f461167c640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f460c0ae6c0 con 0x7f460c0585e0 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f461167c640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f461167c640 10 monclient(hunting): authenticate will time out at 2992.064187s 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f73dcbf8640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f73d80ba350 0x7f73d81f8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45218/0 (socket says 10.20.193.90:45218) 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f73dcbf8640 1 -- 10.20.193.90:0/2213197521 learned_addr learned my addr 10.20.193.90:0/2213197521 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f2ca4e1d640 1 -- 10.20.193.90:0/1287177133 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2c940031d0 con 0x7f2ca00ba350 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f2ca4e1d640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f2ca4e1d640 1 -- 10.20.193.90:0/1287177133 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2c94004990 con 0x7f2ca00ba350 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f2ca4e1d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f2ca4e1d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f2ca4e1d640 10 monclient: dump: 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f2ca80aa640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.464+0000 7f2ca80aa640 4 set_mon_vals no callback set 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.465+0000 7f2ca80aa640 10 monclient: shutdown 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.465+0000 7f2ca80aa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.465+0000 7f2ca80aa640 1 -- 10.20.193.90:0/1287177133 >> v1:10.20.193.90:6789/0 conn(0x7f2ca00ba350 legacy=0x7f2ca0057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.465+0000 7f2ca80aa640 1 -- 10.20.193.90:0/1287177133 shutdown_connections 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.465+0000 7f2ca80aa640 1 -- 10.20.193.90:0/1287177133 >> 10.20.193.90:0/1287177133 conn(0x7f2ca00b7990 msgr2=0x7f2ca00b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f2ca80aa640 1 -- 10.20.193.90:0/1287177133 shutdown_connections 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f2ca80aa640 1 -- 10.20.193.90:0/1287177133 wait complete. 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9103fff640 1 -- 10.20.193.90:0/433540121 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 518031546 0 0) 0x7f912c0a75b0 con 0x7f912c162f10 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9103fff640 1 -- 10.20.193.90:0/433540121 >> v1:10.20.193.94:6789/0 conn(0x7f912c155b20 legacy=0x7f912c0a6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9103fff640 1 -- 10.20.193.90:0/433540121 >> v1:10.20.193.93:6789/0 conn(0x7f912c056c70 legacy=0x7f912c0ab300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9103fff640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9103fff640 1 monclient: found mon.a 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9103fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9103fff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9103fff640 1 -- 10.20.193.90:0/433540121 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c0ac160 con 0x7f912c162f10 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9103fff640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9103fff640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9103fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9131999640 5 monclient: authenticate success, global_id 4521 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9131999640 10 monclient: _renew_subs 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9131999640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9131999640 1 -- 10.20.193.90:0/433540121 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f912c0ac440 con 0x7f912c162f10 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9131999640 10 client.4521.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9131999640 10 monclient: _renew_subs 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9131999640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9131999640 1 -- 10.20.193.90:0/433540121 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f912c0aca00 con 0x7f912c162f10 2026-02-25T15:01:42.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f9131999640 1 librados: init done 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f2ca80aa640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f2ca80aa640 10 monclient: monmap: 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f2ca80aa640 1 librados: starting msgr at 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.466+0000 7f2ca80aa640 1 librados: starting objecter 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f9131999640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f9131999640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f9131999640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f9131999640 1 -- 10.20.193.90:0/433540121 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f912c15cc30 con 0x7f912c162f10 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 1 Processor -- start 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 1 -- start start 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 1 librados: setting wanted keys 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 1 librados: calling monclient init 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 10 monclient: init 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3831798011 0 0) 0x7f789413a0a0 con 0x7f78940b9f80 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f78897fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f78897fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f78897fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f78897fa640 10 monclient(hunting): my global_id is 4530 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7864003610 con 0x7f78940b9f80 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 10 monclient(hunting): picked mon.noname-b con 0x7f2ca00585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 10 monclient(hunting): picked mon.noname-c con 0x7f2ca00b9f80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 10 monclient(hunting): picked mon.noname-a con 0x7f2ca00ba350 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca01febd0 con 0x7f2ca00ba350 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca00b3100 con 0x7f2ca00585e0 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca00b33e0 con 0x7f2ca00b9f80 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.467+0000 7f2ca80aa640 10 monclient(hunting): authenticate will time out at 2992.068008s 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.472+0000 7fe51ad76640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fe51c0b9f80 0x7fe51c163220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45246/0 (socket says 10.20.193.90:45246) 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.472+0000 7fe51ad76640 1 -- 10.20.193.90:0/1781820184 learned_addr learned my addr 10.20.193.90:0/1781820184 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.473+0000 7f73d4ff9640 10 client.?.objecter ms_handle_connect 0x7f73d80ba350 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.473+0000 7f73d4ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.473+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4067836213 0 0) 0x7f0ddc003610 con 0x7f0df01fd410 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.473+0000 7f0dcf7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.473+0000 7f0dcf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.473+0000 7f0dcf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.473+0000 7f0dcf7fe640 10 monclient(hunting): my global_id is 4527 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.473+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0dbc003610 con 0x7f0df01fd410 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.474+0000 7f60a2575640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f60a40b9f80 0x7f60a4163220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45256/0 (socket says 10.20.193.90:45256) 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.474+0000 7f60a2575640 1 -- 10.20.193.90:0/2368341485 learned_addr learned my addr 10.20.193.90:0/2368341485 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.475+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2382464013 0 0) 0x7fcc68003610 con 0x7fcc9c0b9f80 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.475+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcc9c1608f0 con 0x7fcc9c0b9f80 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.475+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fcc840030f0 con 0x7fcc9c0b9f80 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.475+0000 7fcc82ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.475+0000 7fcc82ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.475+0000 7fcc82ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.478 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.479 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.479 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.479 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.479 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.479 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.479 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.475+0000 7fcc82ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.476+0000 7f5923fff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f59240b9f80 0x7f5924163220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45234/0 (socket says 10.20.193.90:45234) 2026-02-25T15:01:42.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.476+0000 7f5923fff640 1 -- 10.20.193.90:0/1861909580 learned_addr learned my addr 10.20.193.90:0/1861909580 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.515+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f25640032a0 con 0x7f2570157780 2026-02-25T15:01:42.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.515+0000 7f2547fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.515+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f25640039b0 con 0x7f2570157780 2026-02-25T15:01:42.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.515+0000 7f2547fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.515+0000 7f2547fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.515+0000 7f2547fff640 10 monclient: dump: 2026-02-25T15:01:42.517 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.517 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.517 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.517 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.517 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.517 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.517 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.517 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.517 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.517 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.515+0000 7f256d7fa640 4 set_mon_vals no callback set 2026-02-25T15:01:42.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.516+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f2564003b90 con 0x7f2570157780 2026-02-25T15:01:42.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.517+0000 7f256e7fc640 10 monclient: get_auth_request con 0x7f2540079f80 auth_method 0 2026-02-25T15:01:42.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.517+0000 7f2547fff640 10 client.4505.objecter ms_handle_connect 0x7f2540079f80 2026-02-25T15:01:42.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.517+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f2564095be0 con 0x7f2570157780 2026-02-25T15:01:42.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.517+0000 7f2547fff640 10 client.4505.objecter ms_dispatch2 0x7f2570000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.517+0000 7f2547fff640 3 client.4505.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.517+0000 7f2547fff640 3 client.4505.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.517+0000 7f2547fff640 20 client.4505.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.519+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f256405f540 con 0x7f2570157780 2026-02-25T15:01:42.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.519+0000 7f2547fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.519+0000 7f2547fff640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:42.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.519+0000 7fafed7fa640 10 client.?.objecter ms_handle_connect 0x7faff80ba350 2026-02-25T15:01:42.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.519+0000 7fafed7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.521+0000 7f44ceffd640 1 -- 10.20.193.90:0/3403664117 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f44bc002fc0 con 0x7f44d0059e70 2026-02-25T15:01:42.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.521+0000 7f44ceffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.521+0000 7f44ceffd640 1 -- 10.20.193.90:0/3403664117 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f44bc003400 con 0x7f44d0059e70 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.521+0000 7f44ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.521+0000 7f44ceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.521+0000 7f44ceffd640 10 monclient: dump: 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.521+0000 7f44d6604640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.521+0000 7f44d6604640 4 set_mon_vals no callback set 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.522+0000 7f44d6604640 10 monclient: shutdown 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.522+0000 7f44d6604640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.522+0000 7f44d6604640 1 -- 10.20.193.90:0/3403664117 >> v1:10.20.193.94:6789/0 conn(0x7f44d0059e70 legacy=0x7f44d00b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.522+0000 7f44d6604640 1 -- 10.20.193.90:0/3403664117 shutdown_connections 2026-02-25T15:01:42.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.522+0000 7f44d6604640 1 -- 10.20.193.90:0/3403664117 >> 10.20.193.90:0/3403664117 conn(0x7f44d00590f0 msgr2=0x7f44d00594f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.522+0000 7f44d6604640 1 -- 10.20.193.90:0/3403664117 shutdown_connections 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.522+0000 7f44d6604640 1 -- 10.20.193.90:0/3403664117 wait complete. 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 10 monclient: monmap: 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 1 librados: starting msgr at 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 1 librados: starting objecter 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 1 Processor -- start 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 1 -- start start 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 1 librados: setting wanted keys 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 1 librados: calling monclient init 2026-02-25T15:01:42.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 10 monclient: init 2026-02-25T15:01:42.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:42.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 10 monclient(hunting): picked mon.noname-a con 0x7f44d0059e70 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 10 monclient(hunting): picked mon.noname-b con 0x7f44d005a240 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 10 monclient(hunting): picked mon.noname-c con 0x7f44d0162e00 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d0055a90 con 0x7f44d0059e70 2026-02-25T15:01:42.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d0055d70 con 0x7f44d005a240 2026-02-25T15:01:42.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d0056050 con 0x7f44d0162e00 2026-02-25T15:01:42.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f44d6604640 10 monclient(hunting): authenticate will time out at 2992.123960s 2026-02-25T15:01:42.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.523+0000 7f4d37fff640 1 -- 10.20.193.90:0/2747018121 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4d3c003000 con 0x7f4d48159e40 2026-02-25T15:01:42.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f4d37fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f4d37fff640 1 -- 10.20.193.90:0/2747018121 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4d3c003400 con 0x7f4d48159e40 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f4d37fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f4d37fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f4d37fff640 10 monclient: dump: 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f4d4f71a640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f4d4f71a640 4 set_mon_vals no callback set 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f78897fa640 10 client.?.objecter ms_handle_connect 0x7f78940585e0 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f78897fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7fcc82ffd640 10 client.?.objecter ms_handle_connect 0x7fcc9c0585e0 2026-02-25T15:01:42.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7fcc82ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f4d4f71a640 10 monclient: shutdown 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f4d4f71a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f4d4f71a640 1 -- 10.20.193.90:0/2747018121 >> v1:10.20.193.94:6789/0 conn(0x7f4d48159e40 legacy=0x7f4d48163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f4d4f71a640 1 -- 10.20.193.90:0/2747018121 shutdown_connections 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f4d4f71a640 1 -- 10.20.193.90:0/2747018121 >> 10.20.193.90:0/2747018121 conn(0x7f4d480b7780 msgr2=0x7f4d480b7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f4d4f71a640 1 -- 10.20.193.90:0/2747018121 shutdown_connections 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.524+0000 7f4d4f71a640 1 -- 10.20.193.90:0/2747018121 wait complete. 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 10 monclient: monmap: 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 1 librados: starting msgr at 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 1 librados: starting objecter 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 1 Processor -- start 2026-02-25T15:01:42.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 1 -- start start 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 1 librados: setting wanted keys 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 1 librados: calling monclient init 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 10 monclient: init 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f73d4ff9640 10 client.?.objecter ms_handle_connect 0x7f73d80b9f80 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f73d4ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 10 monclient(hunting): picked mon.noname-a con 0x7f4d48058070 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 10 monclient(hunting): picked mon.noname-c con 0x7f4d48159a70 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 10 monclient(hunting): picked mon.noname-b con 0x7f4d48159e40 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d4813c320 con 0x7f4d48058070 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d480b9cc0 con 0x7f4d48159e40 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d4813b760 con 0x7f4d48159a70 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.525+0000 7f4d4f71a640 10 monclient(hunting): authenticate will time out at 2992.125854s 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.526+0000 7f0dcf7fe640 10 client.?.objecter ms_handle_connect 0x7f0df00b5da0 2026-02-25T15:01:42.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.526+0000 7f0dcf7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.527+0000 7f460affd640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f460c0585e0 0x7f460c1fe660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54534/0 (socket says 10.20.193.90:54534) 2026-02-25T15:01:42.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.527+0000 7f460affd640 1 -- 10.20.193.90:0/2533946933 learned_addr learned my addr 10.20.193.90:0/2533946933 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.528+0000 7f45eb7fe640 10 client.?.objecter ms_handle_connect 0x7f460c0585e0 2026-02-25T15:01:42.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.528+0000 7f45eb7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.528+0000 7f2ca561e640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f2ca00b9f80 0x7f2ca01fe4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54538/0 (socket says 10.20.193.90:54538) 2026-02-25T15:01:42.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.528+0000 7f2ca561e640 1 -- 10.20.193.90:0/1174542173 learned_addr learned my addr 10.20.193.90:0/1174542173 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.530+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3476991770 0 0) 0x7faff8139520 con 0x7faff80ba350 2026-02-25T15:01:42.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.530+0000 7fafed7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.530+0000 7fafed7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.530+0000 7fafed7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.530+0000 7fafed7fa640 10 monclient(hunting): my global_id is 4523 2026-02-25T15:01:42.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.530+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafc8003610 con 0x7faff80ba350 2026-02-25T15:01:42.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.530+0000 7f2c8e7fc640 10 client.?.objecter ms_handle_connect 0x7f2ca00b9f80 2026-02-25T15:01:42.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.530+0000 7f2c8e7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.534+0000 7f73d4ff9640 1 -- 10.20.193.90:0/2213197521 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1641176155 0 0) 0x7f73d81fd370 con 0x7f73d80b9f80 2026-02-25T15:01:42.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.534+0000 7f73d4ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.534+0000 7f73d4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.534+0000 7f73d4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.534+0000 7f73d4ff9640 10 monclient(hunting): my global_id is 4532 2026-02-25T15:01:42.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.534+0000 7f73d4ff9640 1 -- 10.20.193.90:0/2213197521 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73a8003610 con 0x7f73d80b9f80 2026-02-25T15:01:42.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.537+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 270605101 0 0) 0x7f7894139520 con 0x7f78940585e0 2026-02-25T15:01:42.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.537+0000 7f78897fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.537+0000 7f78897fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.537+0000 7f78897fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.537+0000 7f78897fa640 10 monclient(hunting): my global_id is 4526 2026-02-25T15:01:42.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.537+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f789413a0a0 con 0x7f78940585e0 2026-02-25T15:01:42.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.537+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2590115904 0 0) 0x7f460c0ae6c0 con 0x7f460c0585e0 2026-02-25T15:01:42.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.537+0000 7f45eb7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.537+0000 7f45eb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.537+0000 7f45eb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.537+0000 7f45eb7fe640 10 monclient(hunting): my global_id is 4541 2026-02-25T15:01:42.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.537+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45d8003610 con 0x7f460c0585e0 2026-02-25T15:01:42.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3658635349 0 0) 0x7f0df00b06a0 con 0x7f0df00b5da0 2026-02-25T15:01:42.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f0dcf7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f0dcf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f0dcf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f0dcf7fe640 10 monclient(hunting): my global_id is 4535 2026-02-25T15:01:42.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ddc003610 con 0x7f0df00b5da0 2026-02-25T15:01:42.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3578092402 0 0) 0x7fcc9c160610 con 0x7fcc9c0585e0 2026-02-25T15:01:42.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7fcc82ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7fcc82ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7fcc82ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7fcc82ffd640 10 monclient(hunting): my global_id is 4529 2026-02-25T15:01:42.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc68003610 con 0x7fcc9c0585e0 2026-02-25T15:01:42.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f5922ffd640 1 -- 10.20.193.90:0/1861909580 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2112634197 0 0) 0x7f5924163e90 con 0x7f59240ba350 2026-02-25T15:01:42.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f5922ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f5922ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f5922ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f5922ffd640 10 monclient(hunting): my global_id is 4538 2026-02-25T15:01:42.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f5922ffd640 1 -- 10.20.193.90:0/1861909580 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5908003610 con 0x7f59240ba350 2026-02-25T15:01:42.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7fe519d74640 1 -- 10.20.193.90:0/1781820184 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1955043954 0 0) 0x7fe51c163e90 con 0x7fe51c0585e0 2026-02-25T15:01:42.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7fe519d74640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7fe519d74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7fe519d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7fe519d74640 10 monclient(hunting): my global_id is 4544 2026-02-25T15:01:42.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7fe519d74640 1 -- 10.20.193.90:0/1781820184 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe508003610 con 0x7fe51c0585e0 2026-02-25T15:01:42.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1475802077 0 0) 0x7f2ca00b33e0 con 0x7f2ca00b9f80 2026-02-25T15:01:42.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f2c8e7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f2c8e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f2c8e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f2c8e7fc640 10 monclient(hunting): my global_id is 4547 2026-02-25T15:01:42.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c70003610 con 0x7f2ca00b9f80 2026-02-25T15:01:42.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f44d4b7a640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f44d0162e00 0x7f44d014cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54542/0 (socket says 10.20.193.90:54542) 2026-02-25T15:01:42.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.539+0000 7f44d4b7a640 1 -- 10.20.193.90:0/458283867 learned_addr learned my addr 10.20.193.90:0/458283867 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.542+0000 7f44ccff9640 10 client.?.objecter ms_handle_connect 0x7f44d0162e00 2026-02-25T15:01:42.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.542+0000 7f44ccff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.543+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3085391900 0 0) 0x7fafc8003610 con 0x7faff80ba350 2026-02-25T15:01:42.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.543+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faff8139520 con 0x7faff80ba350 2026-02-25T15:01:42.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.543+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7faff40036d0 con 0x7faff80ba350 2026-02-25T15:01:42.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.543+0000 7fafed7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.543+0000 7fafed7fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.543+0000 7fafed7fa640 10 monclient(hunting): dump: 2026-02-25T15:01:42.544 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.544 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.544 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.545 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.545 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.545 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.545 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.545 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.545 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.545 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.545+0000 7f73d4ff9640 1 -- 10.20.193.90:0/2213197521 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3957384274 0 0) 0x7f73a8003610 con 0x7f73d80b9f80 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.545+0000 7f73d4ff9640 1 -- 10.20.193.90:0/2213197521 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f73d81fd370 con 0x7f73d80b9f80 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.545+0000 7f73d4ff9640 1 -- 10.20.193.90:0/2213197521 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f73c8002ed0 con 0x7f73d80b9f80 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.545+0000 7f73d4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.545+0000 7f73d4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.545+0000 7f73d4ff9640 10 monclient(hunting): dump: 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.545+0000 7f73d4ff9640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.545+0000 7f60a1573640 1 -- 10.20.193.90:0/2368341485 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1534812205 0 0) 0x7f60a4163e90 con 0x7f60a40ba350 2026-02-25T15:01:42.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.545+0000 7f60a1573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.545+0000 7f60a1573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.545+0000 7f60a1573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.545+0000 7f60a1573640 10 monclient(hunting): my global_id is 4550 2026-02-25T15:01:42.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.545+0000 7f60a1573640 1 -- 10.20.193.90:0/2368341485 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6090003610 con 0x7f60a40ba350 2026-02-25T15:01:42.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.548+0000 7f4d4cc8e640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f4d48159a70 0x7f4d480b95b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54558/0 (socket says 10.20.193.90:54558) 2026-02-25T15:01:42.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.548+0000 7f4d4cc8e640 1 -- 10.20.193.90:0/1258207199 learned_addr learned my addr 10.20.193.90:0/1258207199 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.550+0000 7f4d35ffb640 10 client.?.objecter ms_handle_connect 0x7f4d48159a70 2026-02-25T15:01:42.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.550+0000 7f4d35ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1549772912 0 0) 0x7fcc68003610 con 0x7fcc9c0585e0 2026-02-25T15:01:42.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcc9c160610 con 0x7fcc9c0585e0 2026-02-25T15:01:42.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fcc90002fc0 con 0x7fcc9c0585e0 2026-02-25T15:01:42.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7fcc82ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7fcc82ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7fcc82ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:42.553 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.553 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.553 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.553 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.554 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.554 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.554 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.554 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.554 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.554 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f5922ffd640 1 -- 10.20.193.90:0/1861909580 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1645068482 0 0) 0x7f5908003610 con 0x7f59240ba350 2026-02-25T15:01:42.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f5922ffd640 1 -- 10.20.193.90:0/1861909580 >> v1:10.20.193.93:6789/0 conn(0x7f59240585e0 legacy=0x7f5924166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:42.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f5922ffd640 1 -- 10.20.193.90:0/1861909580 >> v1:10.20.193.90:6789/0 conn(0x7f59240b9f80 legacy=0x7f5924163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:42.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f5922ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f5922ffd640 1 monclient: found mon.noname-c 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f5922ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f5922ffd640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f5922ffd640 1 -- 10.20.193.90:0/1861909580 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59240573f0 con 0x7f59240ba350 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f5922ffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f5922ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f5922ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f5922ffd640 1 -- 10.20.193.90:0/1861909580 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5914002df0 con 0x7f59240ba350 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f5922ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f5922ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f5922ffd640 10 monclient: dump: 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f592a5a9640 5 monclient: authenticate success, global_id 4538 2026-02-25T15:01:42.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.552+0000 7f592a5a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.556+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2192700361 0 0) 0x7f45d8003610 con 0x7f460c0585e0 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.556+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f460c0ae6c0 con 0x7f460c0585e0 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.556+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4600002fd0 con 0x7f460c0585e0 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.556+0000 7f45eb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.556+0000 7f45eb7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.556+0000 7f45eb7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.556+0000 7f45eb7fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2628497790 0 0) 0x7faff8139520 con 0x7faff80ba350 2026-02-25T15:01:42.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 >> v1:10.20.193.93:6789/0 conn(0x7faff80b9f80 legacy=0x7faff81656b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 >> v1:10.20.193.90:6789/0 conn(0x7faff80585e0 legacy=0x7faff81624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7fafed7fa640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7fafed7fa640 1 monclient: found mon.c 2026-02-25T15:01:42.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7fafed7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7fafed7fa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff8139800 con 0x7faff80ba350 2026-02-25T15:01:42.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7fafed7fa640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7fafed7fa640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7fafed7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7faffee7b640 5 monclient: authenticate success, global_id 4523 2026-02-25T15:01:42.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7faffee7b640 10 monclient: _renew_subs 2026-02-25T15:01:42.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7faffee7b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7faffee7b640 1 -- 10.20.193.90:0/4251103413 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7faff8139ae0 con 0x7faff80ba350 2026-02-25T15:01:42.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7faffee7b640 10 client.4523.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7faffee7b640 10 monclient: _renew_subs 2026-02-25T15:01:42.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7faffee7b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7faffee7b640 1 -- 10.20.193.90:0/4251103413 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7faff81ff0b0 con 0x7faff80ba350 2026-02-25T15:01:42.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.557+0000 7faffee7b640 1 librados: init done 2026-02-25T15:01:42.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7faffee7b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7faffee7b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7faffee7b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7faffee7b640 1 -- 10.20.193.90:0/4251103413 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff81ff340 con 0x7faff80ba350 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73d4ff9640 1 -- 10.20.193.90:0/2213197521 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4050796056 0 0) 0x7f73d81fd370 con 0x7f73d80b9f80 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73d4ff9640 1 -- 10.20.193.90:0/2213197521 >> v1:10.20.193.93:6789/0 conn(0x7f73d80585e0 legacy=0x7f73d81fb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73d4ff9640 1 -- 10.20.193.90:0/2213197521 >> v1:10.20.193.90:6789/0 conn(0x7f73d80ba350 legacy=0x7f73d81f8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73d4ff9640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73d4ff9640 1 monclient: found mon.c 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73d4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73d4ff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73d4ff9640 1 -- 10.20.193.90:0/2213197521 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d81fd650 con 0x7f73d80b9f80 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73d4ff9640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73d4ff9640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73d4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73de682640 5 monclient: authenticate success, global_id 4532 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73de682640 10 monclient: _renew_subs 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73de682640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73de682640 1 -- 10.20.193.90:0/2213197521 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f73d81fd930 con 0x7f73d80b9f80 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73de682640 10 client.4532.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73de682640 10 monclient: _renew_subs 2026-02-25T15:01:42.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73de682640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73de682640 1 -- 10.20.193.90:0/2213197521 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f73d81f9510 con 0x7f73d80b9f80 2026-02-25T15:01:42.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73de682640 1 librados: init done 2026-02-25T15:01:42.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73de682640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73de682640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73de682640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.558+0000 7f73de682640 1 -- 10.20.193.90:0/2213197521 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d8159080 con 0x7f73d80b9f80 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.563+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 975945095 0 0) 0x7f2c70003610 con 0x7f2ca00b9f80 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.563+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca00b33e0 con 0x7f2ca00b9f80 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.563+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2c94002ed0 con 0x7f2ca00b9f80 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.563+0000 7f2c8e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.563+0000 7f2c8e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.563+0000 7f2c8e7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.563+0000 7f2c8e7fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2607968851 0 0) 0x7fcc9c160610 con 0x7fcc9c0585e0 2026-02-25T15:01:42.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 >> v1:10.20.193.93:6789/0 conn(0x7fcc9c0ba350 legacy=0x7fcc9c1fbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 >> v1:10.20.193.90:6789/0 conn(0x7fcc9c0b9f80 legacy=0x7fcc9c15ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcc82ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcc82ffd640 1 monclient: found mon.c 2026-02-25T15:01:42.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcc82ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcc82ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9c1fcbd0 con 0x7fcc9c0585e0 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcc82ffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcc82ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcc82ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcca0ee8640 5 monclient: authenticate success, global_id 4529 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcca0ee8640 10 monclient: _renew_subs 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcca0ee8640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcca0ee8640 1 -- 10.20.193.90:0/129879591 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9c1fceb0 con 0x7fcc9c0585e0 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcca0ee8640 10 client.4529.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcca0ee8640 10 monclient: _renew_subs 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcca0ee8640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcca0ee8640 1 -- 10.20.193.90:0/129879591 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fcc9c1fd470 con 0x7fcc9c0585e0 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.564+0000 7fcca0ee8640 1 librados: init done 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.565+0000 7fcca0ee8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.565+0000 7fcca0ee8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.565+0000 7fcca0ee8640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.565+0000 7fcca0ee8640 1 -- 10.20.193.90:0/129879591 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc9c158b80 con 0x7fcc9c0585e0 2026-02-25T15:01:42.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f5922ffd640 1 -- 10.20.193.90:0/1861909580 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5914003000 con 0x7f59240ba350 2026-02-25T15:01:42.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f5922ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f5922ffd640 1 -- 10.20.193.90:0/1861909580 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5914003440 con 0x7f59240ba350 2026-02-25T15:01:42.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f5922ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f5922ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f5922ffd640 10 monclient: dump: 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f592a5a9640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f592a5a9640 4 set_mon_vals no callback set 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 107981280 0 0) 0x7f44d0056050 con 0x7f44d0162e00 2026-02-25T15:01:42.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f44ccff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f44ccff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f44ccff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f44ccff9640 10 monclient(hunting): my global_id is 4553 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44a0003610 con 0x7f44d0162e00 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f9103fff640 1 -- 10.20.193.90:0/433540121 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f911c0032f0 con 0x7f912c162f10 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f9103fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f9103fff640 1 -- 10.20.193.90:0/433540121 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f911c003cb0 con 0x7f912c162f10 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f9103fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f9103fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f9103fff640 10 monclient: dump: 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.571 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.572 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.572 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.572 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.572 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f91297fa640 4 set_mon_vals no callback set 2026-02-25T15:01:42.572 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f45eb7fe640 10 client.?.objecter ms_handle_connect 0x7f460c0ba350 2026-02-25T15:01:42.572 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.568+0000 7f45eb7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.572 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.569+0000 7f592a5a9640 10 monclient: shutdown 2026-02-25T15:01:42.572 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.569+0000 7f592a5a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.572 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.569+0000 7f592a5a9640 1 -- 10.20.193.90:0/1861909580 >> v1:10.20.193.94:6789/0 conn(0x7f59240ba350 legacy=0x7f5924057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.572 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.569+0000 7f592a5a9640 1 -- 10.20.193.90:0/1861909580 shutdown_connections 2026-02-25T15:01:42.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.569+0000 7f592a5a9640 1 -- 10.20.193.90:0/1861909580 >> 10.20.193.90:0/1861909580 conn(0x7f59240b7990 msgr2=0x7f59240b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.569+0000 7f592a5a9640 1 -- 10.20.193.90:0/1861909580 shutdown_connections 2026-02-25T15:01:42.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.569+0000 7f592a5a9640 1 -- 10.20.193.90:0/1861909580 wait complete. 2026-02-25T15:01:42.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.569+0000 7f9103fff640 1 -- 10.20.193.90:0/433540121 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f911c003e50 con 0x7f912c162f10 2026-02-25T15:01:42.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 10 monclient: monmap: 2026-02-25T15:01:42.573 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.573 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.573 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.573 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.573 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.573 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.573 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.573 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.573 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.574 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f912affd640 10 monclient: get_auth_request con 0x7f90fc079f30 auth_method 0 2026-02-25T15:01:42.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 1 librados: starting msgr at 2026-02-25T15:01:42.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 1 librados: starting objecter 2026-02-25T15:01:42.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 1 Processor -- start 2026-02-25T15:01:42.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 1 -- start start 2026-02-25T15:01:42.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 1 librados: setting wanted keys 2026-02-25T15:01:42.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 1 librados: calling monclient init 2026-02-25T15:01:42.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 10 monclient: init 2026-02-25T15:01:42.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f9103fff640 10 client.4521.objecter ms_handle_connect 0x7f90fc079f30 2026-02-25T15:01:42.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:42.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 10 monclient(hunting): picked mon.noname-a con 0x7f59240585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 10 monclient(hunting): picked mon.noname-b con 0x7f59240b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 10 monclient(hunting): picked mon.noname-c con 0x7f59240ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5924162b20 con 0x7f59240585e0 2026-02-25T15:01:42.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5924165d10 con 0x7f59240b9f80 2026-02-25T15:01:42.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5924165ff0 con 0x7f59240ba350 2026-02-25T15:01:42.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.570+0000 7f592a5a9640 10 monclient(hunting): authenticate will time out at 2992.170958s 2026-02-25T15:01:42.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.573+0000 7f9103fff640 1 -- 10.20.193.90:0/433540121 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f911c095e50 con 0x7f912c162f10 2026-02-25T15:01:42.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.573+0000 7f9103fff640 10 client.4521.objecter ms_dispatch2 0x7f912c000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.573+0000 7f9103fff640 3 client.4521.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.573+0000 7f9103fff640 3 client.4521.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.573+0000 7f9103fff640 20 client.4521.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.573+0000 7f9103fff640 1 -- 10.20.193.90:0/433540121 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f911c05f7b0 con 0x7f912c162f10 2026-02-25T15:01:42.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.573+0000 7f9103fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.573+0000 7f9103fff640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:42.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.574+0000 7fe519d74640 1 -- 10.20.193.90:0/1781820184 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 245556898 0 0) 0x7fe510002910 con 0x7fe51c0b9f80 2026-02-25T15:01:42.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.574+0000 7fe519d74640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.574+0000 7fe519d74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.574+0000 7fe519d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.574+0000 7fe519d74640 10 monclient(hunting): my global_id is 4536 2026-02-25T15:01:42.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.574+0000 7fe519d74640 1 -- 10.20.193.90:0/1781820184 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe51c163e90 con 0x7fe51c0b9f80 2026-02-25T15:01:42.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.579+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2886554034 0 0) 0x7f7864003610 con 0x7f78940b9f80 2026-02-25T15:01:42.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.579+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7894139520 con 0x7f78940b9f80 2026-02-25T15:01:42.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.579+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f787c002f70 con 0x7f78940b9f80 2026-02-25T15:01:42.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.579+0000 7f78897fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.579+0000 7f78897fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.579+0000 7f78897fa640 10 monclient(hunting): dump: 2026-02-25T15:01:42.580 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.580 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.580 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.580 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.580 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.580 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.580 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.581 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.581 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.581 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.579+0000 7f78897fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.580+0000 7f60a1573640 1 -- 10.20.193.90:0/2368341485 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3240857731 0 0) 0x7f6098002910 con 0x7f60a40b9f80 2026-02-25T15:01:42.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.580+0000 7f60a1573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.580+0000 7f60a1573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.580+0000 7f60a1573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.580+0000 7f60a1573640 10 monclient(hunting): my global_id is 4539 2026-02-25T15:01:42.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.580+0000 7f60a1573640 1 -- 10.20.193.90:0/2368341485 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60a4163e90 con 0x7f60a40b9f80 2026-02-25T15:01:42.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.583+0000 7f2c8e7fc640 10 client.?.objecter ms_handle_connect 0x7f2ca00ba350 2026-02-25T15:01:42.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.583+0000 7f2c8e7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.583+0000 7f44ccff9640 10 client.?.objecter ms_handle_connect 0x7f44d0059e70 2026-02-25T15:01:42.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.583+0000 7f44ccff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.585+0000 7f5923fff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f59240585e0 0x7f5924162410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:45298/0 (socket says 10.20.193.90:45298) 2026-02-25T15:01:42.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.585+0000 7f5923fff640 1 -- 10.20.193.90:0/1776190241 learned_addr learned my addr 10.20.193.90:0/1776190241 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.585+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1964260604 0 0) 0x7f0dbc003610 con 0x7f0df01fd410 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.585+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0df00b06a0 con 0x7f0df01fd410 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.585+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0de0003430 con 0x7f0df01fd410 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.585+0000 7f0dcf7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.585+0000 7f0dcf7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.585+0000 7f0dcf7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.585+0000 7f0dcf7fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.588 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.586+0000 7f5920ff9640 10 client.?.objecter ms_handle_connect 0x7f59240585e0 2026-02-25T15:01:42.588 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.586+0000 7f5920ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.588 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.586+0000 7f4d35ffb640 10 client.?.objecter ms_handle_connect 0x7f4d48058070 2026-02-25T15:01:42.588 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.586+0000 7f4d35ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.590 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.589+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2596388776 0 0) 0x7f460c1fed70 con 0x7f460c0ba350 2026-02-25T15:01:42.590 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.589+0000 7f45eb7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.590 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.589+0000 7f45eb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.590 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.589+0000 7f45eb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.590 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.589+0000 7f45eb7fe640 10 monclient(hunting): my global_id is 4542 2026-02-25T15:01:42.590 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.589+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45d8003610 con 0x7f460c0ba350 2026-02-25T15:01:42.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 443400230 0 0) 0x7f44d0055a90 con 0x7f44d0059e70 2026-02-25T15:01:42.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7f44ccff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7f44ccff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7f44ccff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7f44ccff9640 10 monclient(hunting): my global_id is 4545 2026-02-25T15:01:42.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44d0056050 con 0x7f44d0059e70 2026-02-25T15:01:42.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe519d74640 1 -- 10.20.193.90:0/1781820184 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1512872554 0 0) 0x7fe51c163e90 con 0x7fe51c0b9f80 2026-02-25T15:01:42.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe519d74640 1 -- 10.20.193.90:0/1781820184 >> v1:10.20.193.94:6789/0 conn(0x7fe51c0585e0 legacy=0x7fe51c166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe519d74640 1 -- 10.20.193.90:0/1781820184 >> v1:10.20.193.93:6789/0 conn(0x7fe51c0ba350 legacy=0x7fe51c057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe519d74640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe519d74640 1 monclient: found mon.noname-a 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe519d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe519d74640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe519d74640 1 -- 10.20.193.90:0/1781820184 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c0573f0 con 0x7fe51c0b9f80 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe519d74640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe519d74640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe519d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe519d74640 1 -- 10.20.193.90:0/1781820184 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe510002fc0 con 0x7fe51c0b9f80 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe519d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe519d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe519d74640 10 monclient: dump: 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe52125b640 5 monclient: authenticate success, global_id 4536 2026-02-25T15:01:42.597 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.594+0000 7fe52125b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.597 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2965557319 0 0) 0x7f4d4813c320 con 0x7f4d48058070 2026-02-25T15:01:42.597 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f4d35ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.597 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f4d35ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.597 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f4d35ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.597 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f4d35ffb640 10 monclient(hunting): my global_id is 4551 2026-02-25T15:01:42.597 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d18003610 con 0x7f4d48058070 2026-02-25T15:01:42.598 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 27796886 0 0) 0x7f2ca01febd0 con 0x7f2ca00ba350 2026-02-25T15:01:42.598 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f2c8e7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.598 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f2c8e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.598 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f2c8e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.598 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f2c8e7fc640 10 monclient(hunting): my global_id is 4548 2026-02-25T15:01:42.598 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c70003610 con 0x7f2ca00ba350 2026-02-25T15:01:42.598 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f60a1573640 1 -- 10.20.193.90:0/2368341485 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2933943005 0 0) 0x7f60a4163e90 con 0x7f60a40b9f80 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f60a1573640 1 -- 10.20.193.90:0/2368341485 >> v1:10.20.193.94:6789/0 conn(0x7f60a40ba350 legacy=0x7f60a4057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f60a1573640 1 -- 10.20.193.90:0/2368341485 >> v1:10.20.193.93:6789/0 conn(0x7f60a40585e0 legacy=0x7f60a4166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f60a1573640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f60a1573640 1 monclient: found mon.noname-a 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f60a1573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f60a1573640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f60a1573640 1 -- 10.20.193.90:0/2368341485 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a40573f0 con 0x7f60a40b9f80 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f60a1573640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f60a1573640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f60a1573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f60a1573640 1 -- 10.20.193.90:0/2368341485 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6098002fc0 con 0x7f60a40b9f80 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f60a1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f60a1573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.596+0000 7f60a1573640 10 monclient: dump: 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.599 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.597+0000 7f60a8dc8640 5 monclient: authenticate success, global_id 4539 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.597+0000 7f60a8dc8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.597+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 206771682 0 0) 0x7f45d8003610 con 0x7f460c0ba350 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.597+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f460c1fed70 con 0x7f460c0ba350 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.597+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f45fc0032c0 con 0x7f460c0ba350 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.597+0000 7f45eb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.597+0000 7f45eb7fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.597+0000 7f45eb7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.600 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.601 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.601 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.601 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.601 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.601 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.597+0000 7f5920ff9640 1 -- 10.20.193.90:0/1776190241 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1600439239 0 0) 0x7f5924162b20 con 0x7f59240585e0 2026-02-25T15:01:42.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.597+0000 7f5920ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.597+0000 7f5920ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.597+0000 7f5920ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.597+0000 7f5920ff9640 10 monclient(hunting): my global_id is 4554 2026-02-25T15:01:42.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.597+0000 7f5920ff9640 1 -- 10.20.193.90:0/1776190241 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58f4003610 con 0x7f59240585e0 2026-02-25T15:01:42.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1485652197 0 0) 0x7f0df00b06a0 con 0x7f0df01fd410 2026-02-25T15:01:42.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 >> v1:10.20.193.94:6789/0 conn(0x7f0df00b5da0 legacy=0x7f0df015fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 >> v1:10.20.193.93:6789/0 conn(0x7f0df015b0a0 legacy=0x7f0df01fcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0dcf7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0dcf7fe640 1 monclient: found mon.a 2026-02-25T15:01:42.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0dcf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0dcf7fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df00b0980 con 0x7f0df01fd410 2026-02-25T15:01:42.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0dcf7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0dcf7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0dcf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0df5171640 5 monclient: authenticate success, global_id 4527 2026-02-25T15:01:42.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0df5171640 10 monclient: _renew_subs 2026-02-25T15:01:42.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0df5171640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0df5171640 1 -- 10.20.193.90:0/3673364815 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0df00b0c60 con 0x7f0df01fd410 2026-02-25T15:01:42.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0df5171640 10 client.4527.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0df5171640 10 monclient: _renew_subs 2026-02-25T15:01:42.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0df5171640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0df5171640 1 -- 10.20.193.90:0/3673364815 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0df00b1220 con 0x7f0df01fd410 2026-02-25T15:01:42.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.600+0000 7f0df5171640 1 librados: init done 2026-02-25T15:01:42.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.601+0000 7f0df5171640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.601+0000 7f0df5171640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.601+0000 7f0df5171640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.601+0000 7f0df5171640 1 -- 10.20.193.90:0/3673364815 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df00525e0 con 0x7f0df01fd410 2026-02-25T15:01:42.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 963473409 0 0) 0x7f4d18003610 con 0x7f4d48058070 2026-02-25T15:01:42.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4d4813c320 con 0x7f4d48058070 2026-02-25T15:01:42.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3847999741 0 0) 0x7f44d0056050 con 0x7f44d0059e70 2026-02-25T15:01:42.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f44d0055a90 con 0x7f44d0059e70 2026-02-25T15:01:42.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4d38004100 con 0x7f4d48058070 2026-02-25T15:01:42.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f4d35ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f4d35ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f4d35ffb640 10 monclient(hunting): dump: 2026-02-25T15:01:42.604 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.605 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.605 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.605 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.605 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.605 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.605 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.605 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.605 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.605 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f4d35ffb640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3138790020 0 0) 0x7f7894139520 con 0x7f78940b9f80 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 >> v1:10.20.193.94:6789/0 conn(0x7f78940585e0 legacy=0x7f78941624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 >> v1:10.20.193.93:6789/0 conn(0x7f78940ba350 legacy=0x7f789425f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f78897fa640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f78897fa640 1 monclient: found mon.a 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f78897fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f78897fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7894139800 con 0x7f78940b9f80 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f78897fa640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f78897fa640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f78897fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f789b18a640 5 monclient: authenticate success, global_id 4530 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f789b18a640 10 monclient: _renew_subs 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f789b18a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f789b18a640 1 -- 10.20.193.90:0/3904297474 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7894139ae0 con 0x7f78940b9f80 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f789b18a640 10 client.4530.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f789b18a640 10 monclient: _renew_subs 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f789b18a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f789b18a640 1 -- 10.20.193.90:0/3904297474 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f78941ff0b0 con 0x7f78940b9f80 2026-02-25T15:01:42.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f44bc003370 con 0x7f44d0059e70 2026-02-25T15:01:42.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f44ccff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f789b18a640 1 librados: init done 2026-02-25T15:01:42.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f44ccff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f44ccff9640 10 monclient(hunting): dump: 2026-02-25T15:01:42.607 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.607 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.607 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.607 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.607 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.607 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.607 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.607 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.607 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.608 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.608 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.603+0000 7f44ccff9640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.608 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.604+0000 7f789b18a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.608 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.604+0000 7f789b18a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.608 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.604+0000 7f789b18a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.608 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.604+0000 7f789b18a640 1 -- 10.20.193.90:0/3904297474 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78940b64c0 con 0x7f78940b9f80 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7fe519d74640 1 -- 10.20.193.90:0/1781820184 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe5100031d0 con 0x7fe51c0b9f80 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7fe519d74640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7fe519d74640 1 -- 10.20.193.90:0/1781820184 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe510004990 con 0x7fe51c0b9f80 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7fe519d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7fe519d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7fe519d74640 10 monclient: dump: 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7fe52125b640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7fe52125b640 4 set_mon_vals no callback set 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7fe52125b640 10 monclient: shutdown 2026-02-25T15:01:42.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7fe52125b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7fe52125b640 1 -- 10.20.193.90:0/1781820184 >> v1:10.20.193.90:6789/0 conn(0x7fe51c0b9f80 legacy=0x7fe51c163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7fe52125b640 1 -- 10.20.193.90:0/1781820184 shutdown_connections 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7f60a1573640 1 -- 10.20.193.90:0/2368341485 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f60980031d0 con 0x7f60a40b9f80 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7f60a1573640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7f60a1573640 1 -- 10.20.193.90:0/2368341485 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6098004990 con 0x7f60a40b9f80 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7f60a1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7f60a1573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7f60a1573640 10 monclient: dump: 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.610 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.611 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7f60a8dc8640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:42.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7f60a8dc8640 4 set_mon_vals no callback set 2026-02-25T15:01:42.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.606+0000 7fe52125b640 1 -- 10.20.193.90:0/1781820184 >> 10.20.193.90:0/1781820184 conn(0x7fe51c0b7990 msgr2=0x7fe51c0b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.607+0000 7fe52125b640 1 -- 10.20.193.90:0/1781820184 shutdown_connections 2026-02-25T15:01:42.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.607+0000 7fe52125b640 1 -- 10.20.193.90:0/1781820184 wait complete. 2026-02-25T15:01:42.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.607+0000 7f60a8dc8640 10 monclient: shutdown 2026-02-25T15:01:42.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.607+0000 7f60a8dc8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.607+0000 7f60a8dc8640 1 -- 10.20.193.90:0/2368341485 >> v1:10.20.193.90:6789/0 conn(0x7f60a40b9f80 legacy=0x7f60a4163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.607+0000 7f60a8dc8640 1 -- 10.20.193.90:0/2368341485 shutdown_connections 2026-02-25T15:01:42.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.607+0000 7f60a8dc8640 1 -- 10.20.193.90:0/2368341485 >> 10.20.193.90:0/2368341485 conn(0x7f60a40b7990 msgr2=0x7f60a40b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.612 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.607+0000 7f60a8dc8640 1 -- 10.20.193.90:0/2368341485 shutdown_connections 2026-02-25T15:01:42.612 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.607+0000 7f60a8dc8640 1 -- 10.20.193.90:0/2368341485 wait complete. 2026-02-25T15:01:42.612 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.607+0000 7fe52125b640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.612 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.607+0000 7fe52125b640 10 monclient: monmap: 2026-02-25T15:01:42.612 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.612 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.612 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.612 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.612 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.612 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.612 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.612 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.612 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.612 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.607+0000 7f60a8dc8640 10 monclient: build_initial_monmap 2026-02-25T15:01:42.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.607+0000 7f60a8dc8640 10 monclient: monmap: 2026-02-25T15:01:42.613 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:42.613 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:42.613 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:42.613 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:42.613 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:42.613 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:42.613 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:42.613 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:42.613 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:42.613 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.607+0000 7fe52125b640 1 librados: starting msgr at 2026-02-25T15:01:42.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.607+0000 7fe52125b640 1 librados: starting objecter 2026-02-25T15:01:42.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.614 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7feb54003780 con 0x7feb6c0b9f80 2026-02-25T15:01:42.614 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7feb617fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.614 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 1 Processor -- start 2026-02-25T15:01:42.614 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7feb54004cf0 con 0x7feb6c0b9f80 2026-02-25T15:01:42.614 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7feb617fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.614 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7feb617fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.614 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7feb617fa640 10 monclient: dump: 2026-02-25T15:01:42.614 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.614 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.614 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.614 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.614 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.614 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.614 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.614 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.615 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.615 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7feb62ffd640 4 set_mon_vals no callback set 2026-02-25T15:01:42.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 1 librados: starting msgr at 2026-02-25T15:01:42.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 1 librados: starting objecter 2026-02-25T15:01:42.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 1 -- start start 2026-02-25T15:01:42.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 1 librados: setting wanted keys 2026-02-25T15:01:42.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 1 librados: calling monclient init 2026-02-25T15:01:42.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 10 monclient: init 2026-02-25T15:01:42.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:42.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 1 Processor -- start 2026-02-25T15:01:42.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 1 -- start start 2026-02-25T15:01:42.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 1 librados: setting wanted keys 2026-02-25T15:01:42.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 1 librados: calling monclient init 2026-02-25T15:01:42.616 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 10 monclient: init 2026-02-25T15:01:42.616 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.616 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:42.616 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 10 monclient(hunting): picked mon.noname-a con 0x7fe51c0585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.616 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 10 monclient(hunting): picked mon.noname-b con 0x7fe51c0b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.616 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 10 monclient(hunting): picked mon.noname-c con 0x7fe51c0ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.616 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe51c1fc940 con 0x7fe51c0585e0 2026-02-25T15:01:42.616 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe51c1fcc20 con 0x7fe51c0b9f80 2026-02-25T15:01:42.616 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe51c1fcf00 con 0x7fe51c0ba350 2026-02-25T15:01:42.616 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.616 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7fe52125b640 10 monclient(hunting): authenticate will time out at 2992.208568s 2026-02-25T15:01:42.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:42.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:42.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 10 monclient(hunting): picked mon.noname-c con 0x7f60a40585e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:42.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 10 monclient(hunting): picked mon.noname-b con 0x7f60a40b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:42.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 10 monclient(hunting): picked mon.noname-a con 0x7f60a40ba350 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:42.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a40af860 con 0x7f60a40ba350 2026-02-25T15:01:42.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a40afb40 con 0x7f60a40b9f80 2026-02-25T15:01:42.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a40afe20 con 0x7f60a40585e0 2026-02-25T15:01:42.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:42.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7f60a8dc8640 10 monclient(hunting): authenticate will time out at 2992.208651s 2026-02-25T15:01:42.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.608+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7feb5401e1d0 con 0x7feb6c0b9f80 2026-02-25T15:01:42.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.609+0000 7feb70a00640 10 monclient: get_auth_request con 0x7feb3c079fb0 auth_method 0 2026-02-25T15:01:42.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.609+0000 7feb617fa640 10 client.4513.objecter ms_handle_connect 0x7feb3c079fb0 2026-02-25T15:01:42.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.610+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7feb54095ca0 con 0x7feb6c0b9f80 2026-02-25T15:01:42.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.610+0000 7feb617fa640 10 client.4513.objecter ms_dispatch2 0x7feb6c000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.610+0000 7feb617fa640 3 client.4513.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.610+0000 7feb617fa640 3 client.4513.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.610+0000 7feb617fa640 20 client.4513.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2524702429 0 0) 0x7f460c1fed70 con 0x7f460c0ba350 2026-02-25T15:01:42.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 >> v1:10.20.193.94:6789/0 conn(0x7f460c0585e0 legacy=0x7f460c1fe660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 >> v1:10.20.193.93:6789/0 conn(0x7f460c0b9f80 legacy=0x7f460c0af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f45eb7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f45eb7fe640 1 monclient: found mon.a 2026-02-25T15:01:42.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f45eb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f45eb7fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460c0ae9a0 con 0x7f460c0ba350 2026-02-25T15:01:42.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f45eb7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f45eb7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f45eb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f461167c640 5 monclient: authenticate success, global_id 4542 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f461167c640 10 monclient: _renew_subs 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f461167c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f461167c640 1 -- 10.20.193.90:0/2533946933 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f460c0aec80 con 0x7f460c0ba350 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f461167c640 10 client.4542.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f461167c640 10 monclient: _renew_subs 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f461167c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f461167c640 1 -- 10.20.193.90:0/2533946933 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f460c165f10 con 0x7f460c0ba350 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7f461167c640 1 librados: init done 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7feb5405f600 con 0x7feb6c0b9f80 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7feb617fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.611+0000 7feb617fa640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.612+0000 7f461167c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.612+0000 7f461167c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.612+0000 7f461167c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.612+0000 7f461167c640 1 -- 10.20.193.90:0/2533946933 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f460c0b6c80 con 0x7f460c0ba350 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.617+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb4140032a0 con 0x7fb420157780 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.617+0000 7fb3ff7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.617+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb4140039b0 con 0x7fb420157780 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.617+0000 7fb3ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.617+0000 7fb3ff7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.617+0000 7fb3ff7fe640 10 monclient: dump: 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.617+0000 7fb41d573640 4 set_mon_vals no callback set 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.618+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fb414003b90 con 0x7fb420157780 2026-02-25T15:01:42.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.619+0000 7fb41e575640 10 monclient: get_auth_request con 0x7fb3ec07a0f0 auth_method 0 2026-02-25T15:01:42.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.619+0000 7fb3ff7fe640 10 client.4501.objecter ms_handle_connect 0x7fb3ec07a0f0 2026-02-25T15:01:42.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.619+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7fb414095c00 con 0x7fb420157780 2026-02-25T15:01:42.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.619+0000 7fb3ff7fe640 10 client.4501.objecter ms_dispatch2 0x7fb420000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.619+0000 7fb3ff7fe640 3 client.4501.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.619+0000 7fb3ff7fe640 3 client.4501.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.619+0000 7fb3ff7fe640 20 client.4501.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.621+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fb41405f560 con 0x7fb420157780 2026-02-25T15:01:42.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.621+0000 7fb3ff7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.621+0000 7fb3ff7fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:42.627 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.626+0000 7f44ccff9640 10 client.?.objecter ms_handle_connect 0x7f44d005a240 2026-02-25T15:01:42.627 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.626+0000 7f44ccff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.628+0000 7f5920ff9640 10 client.?.objecter ms_handle_connect 0x7f59240b9f80 2026-02-25T15:01:42.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.628+0000 7f5920ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.628+0000 7f4d35ffb640 10 client.?.objecter ms_handle_connect 0x7f4d48159e40 2026-02-25T15:01:42.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.628+0000 7f4d35ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.628+0000 7f2c8e7fc640 10 client.?.objecter ms_handle_connect 0x7f2ca00585e0 2026-02-25T15:01:42.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.628+0000 7f2c8e7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.629+0000 7f257584d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-25T15:01:42.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.629+0000 7f257584d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-25T15:01:42.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.629+0000 7f257584d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.629+0000 7f257584d640 1 -- 10.20.193.90:0/359940460 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f257013d6e0 con 0x7f2570157780 2026-02-25T15:01:42.631 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.630+0000 7fe51a575640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fe51c0b9f80 0x7fe51c1fdcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46856/0 (socket says 10.20.193.90:46856) 2026-02-25T15:01:42.631 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.630+0000 7fe51a575640 1 -- 10.20.193.90:0/4205025154 learned_addr learned my addr 10.20.193.90:0/4205025154 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.631 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.630+0000 7f60a1d74640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f60a40b9f80 0x7f60a41fc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46872/0 (socket says 10.20.193.90:46872) 2026-02-25T15:01:42.631 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.631+0000 7f60a1d74640 1 -- 10.20.193.90:0/2902635105 learned_addr learned my addr 10.20.193.90:0/2902635105 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:42.632 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.631+0000 7fe4fb7fe640 10 client.?.objecter ms_handle_connect 0x7fe51c0b9f80 2026-02-25T15:01:42.632 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.631+0000 7fe4fb7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.632 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.631+0000 7f608affd640 10 client.?.objecter ms_handle_connect 0x7f60a40b9f80 2026-02-25T15:01:42.632 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.631+0000 7f608affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.691 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.690+0000 7f9131999640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-25T15:01:42.691 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.690+0000 7f9131999640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-25T15:01:42.691 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.690+0000 7f9131999640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.691 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.690+0000 7f9131999640 1 -- 10.20.193.90:0/433540121 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f912c154b10 con 0x7f912c162f10 2026-02-25T15:01:42.723 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.722+0000 7feb72c8b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-25T15:01:42.723 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.722+0000 7feb72c8b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-25T15:01:42.723 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.722+0000 7feb72c8b640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.723 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.722+0000 7feb72c8b640 1 -- 10.20.193.90:0/3231812946 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7feb6c0b64c0 con 0x7feb6c0b9f80 2026-02-25T15:01:42.727 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.726+0000 7feb617fa640 1 -- 10.20.193.90:0/3231812946 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (unknown 832126871 0 778662779) 0x7feb5405ef00 con 0x7feb6c0b9f80 2026-02-25T15:01:42.727 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.726+0000 7feb617fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-25T15:01:42.727 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.726+0000 7feb617fa640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:42.727 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:42.730 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.728+0000 7feb72c8b640 10 librados: watch_flush enter 2026-02-25T15:01:42.730 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.728+0000 7feb72c8b640 10 librados: watch_flush exit 2026-02-25T15:01:42.730 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.728+0000 7feb72c8b640 20 client.4513.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.730 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.728+0000 7feb72c8b640 10 client.4513.objecter successfully canceled tick 2026-02-25T15:01:42.730 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.728+0000 7feb72c8b640 1 -- 10.20.193.90:0/3231812946 >> v1:10.20.193.189:6800/3647375982 conn(0x7feb3c079fb0 legacy=0x7feb3c09a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.730 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.728+0000 7feb72c8b640 10 monclient: shutdown 2026-02-25T15:01:42.730 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.728+0000 7feb72c8b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.730 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.728+0000 7feb72c8b640 1 -- 10.20.193.90:0/3231812946 >> v1:10.20.193.93:6789/0 conn(0x7feb6c0b9f80 legacy=0x7feb6c165920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.731 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.728+0000 7feb72c8b640 1 -- 10.20.193.90:0/3231812946 shutdown_connections 2026-02-25T15:01:42.731 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.728+0000 7feb72c8b640 1 -- 10.20.193.90:0/3231812946 >> 10.20.193.90:0/3231812946 conn(0x7feb6c0b7990 msgr2=0x7feb6c0568a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.731 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.728+0000 7feb72c8b640 1 -- 10.20.193.90:0/3231812946 shutdown_connections 2026-02-25T15:01:42.731 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.728+0000 7feb72c8b640 1 -- 10.20.193.90:0/3231812946 wait complete. 2026-02-25T15:01:42.731 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.728+0000 7feb72c8b640 1 librados: shutdown 2026-02-25T15:01:42.732 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.731+0000 7fb425082640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-25T15:01:42.732 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.731+0000 7fb425082640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-25T15:01:42.732 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.731+0000 7fb425082640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:42.733 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.731+0000 7fb425082640 1 -- 10.20.193.90:0/2259846989 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb42015fe40 con 0x7fb420157780 2026-02-25T15:01:42.734 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.733+0000 7fb3ff7fe640 1 -- 10.20.193.90:0/2259846989 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (unknown 574334944 0 778662779) 0x7fb41405ee60 con 0x7fb420157780 2026-02-25T15:01:42.735 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.733+0000 7fb3ff7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-25T15:01:42.735 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.733+0000 7fb3ff7fe640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:42.735 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:42.738 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.735+0000 7fb425082640 10 librados: watch_flush enter 2026-02-25T15:01:42.738 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.735+0000 7fb425082640 10 librados: watch_flush exit 2026-02-25T15:01:42.738 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.735+0000 7fb425082640 20 client.4501.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.738 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.735+0000 7fb425082640 10 client.4501.objecter successfully canceled tick 2026-02-25T15:01:42.738 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.735+0000 7fb425082640 1 -- 10.20.193.90:0/2259846989 >> v1:10.20.193.189:6800/3647375982 conn(0x7fb3ec07a0f0 legacy=0x7fb3ec09a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.738 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.735+0000 7fb425082640 10 monclient: shutdown 2026-02-25T15:01:42.738 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.735+0000 7fb425082640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.738 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.735+0000 7fb425082640 1 -- 10.20.193.90:0/2259846989 >> v1:10.20.193.93:6789/0 conn(0x7fb420157780 legacy=0x7fb4200a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.739 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.736+0000 7fb425082640 1 -- 10.20.193.90:0/2259846989 shutdown_connections 2026-02-25T15:01:42.739 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.736+0000 7fb425082640 1 -- 10.20.193.90:0/2259846989 >> 10.20.193.90:0/2259846989 conn(0x7fb4200b2ef0 msgr2=0x7fb4200b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.739 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.736+0000 7fb425082640 1 -- 10.20.193.90:0/2259846989 shutdown_connections 2026-02-25T15:01:42.739 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.736+0000 7fb425082640 1 -- 10.20.193.90:0/2259846989 wait complete. 2026-02-25T15:01:42.739 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.736+0000 7fb425082640 1 librados: shutdown 2026-02-25T15:01:42.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.1 2026-02-25T15:01:42.741 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.738+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2955204905 0 0) 0x7f4d4813b760 con 0x7f4d48159a70 2026-02-25T15:01:42.741 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.738+0000 7f4d35ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.741 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.738+0000 7f4d35ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.741 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.738+0000 7f4d35ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.741 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.738+0000 7f4d35ffb640 10 monclient(hunting): my global_id is 4556 2026-02-25T15:01:42.741 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.738+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d18003610 con 0x7f4d48159a70 2026-02-25T15:01:42.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.743+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7faff4003890 con 0x7faff80ba350 2026-02-25T15:01:42.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.743+0000 7fafed7fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.743+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7faff4003330 con 0x7faff80ba350 2026-02-25T15:01:42.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.743+0000 7fafed7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.743+0000 7fafed7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.743+0000 7fafeeffd640 4 set_mon_vals no callback set 2026-02-25T15:01:42.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.743+0000 7fafed7fa640 10 monclient: dump: 2026-02-25T15:01:42.745 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.745 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.745 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.745 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.745 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.745 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.745 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.745 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.746 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.746 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.744+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7faff401d110 con 0x7faff80ba350 2026-02-25T15:01:42.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.745+0000 7faffcbf0640 10 monclient: get_auth_request con 0x7fafc8079ec0 auth_method 0 2026-02-25T15:01:42.747 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.745+0000 7fafed7fa640 10 client.4523.objecter ms_handle_connect 0x7fafc8079ec0 2026-02-25T15:01:42.748 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.0 2026-02-25T15:01:42.748 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.745+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7faff4094c70 con 0x7faff80ba350 2026-02-25T15:01:42.748 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.745+0000 7fafed7fa640 10 client.4523.objecter ms_dispatch2 0x7faff8000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.748 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.745+0000 7fafed7fa640 3 client.4523.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.748 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.745+0000 7fafed7fa640 3 client.4523.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.749 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.745+0000 7fafed7fa640 20 client.4523.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3241593085 0 0) 0x7f2c70003610 con 0x7f2ca00ba350 2026-02-25T15:01:42.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca01febd0 con 0x7f2ca00ba350 2026-02-25T15:01:42.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2c9c003240 con 0x7f2ca00ba350 2026-02-25T15:01:42.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f2c8e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f2c8e7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.795 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f2c8e7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:42.795 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.795 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.795 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.795 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.796 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.796 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.796 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.796 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.796 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.796 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.797 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 724721912 0 0) 0x7f44d0055a90 con 0x7f44d0059e70 2026-02-25T15:01:42.797 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 >> v1:10.20.193.94:6789/0 conn(0x7f44d0162e00 legacy=0x7f44d014cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.797 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 >> v1:10.20.193.93:6789/0 conn(0x7f44d005a240 legacy=0x7f44d0153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.797 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44ccff9640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.797 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44ccff9640 1 monclient: found mon.a 2026-02-25T15:01:42.797 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44ccff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.798 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44ccff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.798 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d0056330 con 0x7f44d0059e70 2026-02-25T15:01:42.798 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44ccff9640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.798 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44ccff9640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.798 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44ccff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.798 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44d6604640 5 monclient: authenticate success, global_id 4545 2026-02-25T15:01:42.798 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44d6604640 10 monclient: _renew_subs 2026-02-25T15:01:42.798 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44d6604640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.798 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44d6604640 1 -- 10.20.193.90:0/458283867 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f44d0056610 con 0x7f44d0059e70 2026-02-25T15:01:42.798 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44d6604640 10 client.4545.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.798 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44d6604640 10 monclient: _renew_subs 2026-02-25T15:01:42.798 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44d6604640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.799 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44d6604640 1 -- 10.20.193.90:0/458283867 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f44d01fd780 con 0x7f44d0059e70 2026-02-25T15:01:42.799 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.793+0000 7f44d6604640 1 librados: init done 2026-02-25T15:01:42.799 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f5920ff9640 1 -- 10.20.193.90:0/1776190241 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1756178419 0 0) 0x7f58f4003610 con 0x7f59240585e0 2026-02-25T15:01:42.799 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f5920ff9640 1 -- 10.20.193.90:0/1776190241 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5924162b20 con 0x7f59240585e0 2026-02-25T15:01:42.799 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f5920ff9640 1 -- 10.20.193.90:0/1776190241 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5910002cc0 con 0x7f59240585e0 2026-02-25T15:01:42.800 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f5920ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:42.800 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f5920ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:42.800 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f5920ff9640 10 monclient(hunting): dump: 2026-02-25T15:01:42.800 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.800 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.800 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.800 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.800 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.800 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.800 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.800 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.800 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.801 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.801 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f5920ff9640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:42.801 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f44d6604640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.801 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f44d6604640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.801 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f44d6604640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.801 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f44d6604640 1 -- 10.20.193.90:0/458283867 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d0162310 con 0x7f44d0059e70 2026-02-25T15:01:42.803 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 444489674 0 0) 0x7f4d4813c320 con 0x7f4d48058070 2026-02-25T15:01:42.803 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 >> v1:10.20.193.94:6789/0 conn(0x7f4d48159a70 legacy=0x7f4d480b95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.803 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 >> v1:10.20.193.93:6789/0 conn(0x7f4d48159e40 legacy=0x7f4d4813cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.803 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d35ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.803 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d35ffb640 1 monclient: found mon.a 2026-02-25T15:01:42.803 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d35ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.803 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d35ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.803 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4813ba40 con 0x7f4d48058070 2026-02-25T15:01:42.803 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d35ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.804 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d35ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.804 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d35ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.804 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d4f71a640 5 monclient: authenticate success, global_id 4551 2026-02-25T15:01:42.804 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d4f71a640 10 monclient: _renew_subs 2026-02-25T15:01:42.804 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d4f71a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.804 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d4f71a640 1 -- 10.20.193.90:0/1258207199 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4813bd20 con 0x7f4d48058070 2026-02-25T15:01:42.804 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d4f71a640 10 client.4551.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.804 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d4f71a640 10 monclient: _renew_subs 2026-02-25T15:01:42.804 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d4f71a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.804 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d4f71a640 1 -- 10.20.193.90:0/1258207199 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4d4813c000 con 0x7f4d48058070 2026-02-25T15:01:42.805 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.794+0000 7f4d4f71a640 1 librados: init done 2026-02-25T15:01:42.805 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.795+0000 7fe4fb7fe640 10 client.?.objecter ms_handle_connect 0x7fe51c0585e0 2026-02-25T15:01:42.805 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.795+0000 7fe4fb7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.805 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.795+0000 7f4d4f71a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.806 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.795+0000 7f4d4f71a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.806 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.795+0000 7f4d4f71a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:42.806 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.795+0000 7f4d4f71a640 1 -- 10.20.193.90:0/1258207199 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d48161660 con 0x7f4d48058070 2026-02-25T15:01:42.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.835+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7faff405e5d0 con 0x7faff80ba350 2026-02-25T15:01:42.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.835+0000 7fafed7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.835+0000 7fafed7fa640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:42.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.839+0000 7f73d4ff9640 1 -- 10.20.193.90:0/2213197521 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f73c8003590 con 0x7f73d80b9f80 2026-02-25T15:01:42.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.839+0000 7f73d4ff9640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.840+0000 7f73d4ff9640 1 -- 10.20.193.90:0/2213197521 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f73c80038e0 con 0x7f73d80b9f80 2026-02-25T15:01:42.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.840+0000 7f73d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.840+0000 7f73d4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.840+0000 7f73d4ff9640 10 monclient: dump: 2026-02-25T15:01:42.842 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.842 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.842 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.842 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.842 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.842 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.843 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.843 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.843 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.843 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.840+0000 7f73d67fc640 4 set_mon_vals no callback set 2026-02-25T15:01:42.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.840+0000 7f73d4ff9640 1 -- 10.20.193.90:0/2213197521 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f73c801cdc0 con 0x7f73d80b9f80 2026-02-25T15:01:42.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.841+0000 7f73d7fff640 10 monclient: get_auth_request con 0x7f73a8079e50 auth_method 0 2026-02-25T15:01:42.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.841+0000 7f73d4ff9640 10 client.4532.objecter ms_handle_connect 0x7f73a8079e50 2026-02-25T15:01:42.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.841+0000 7f73d4ff9640 1 -- 10.20.193.90:0/2213197521 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f73c8094ac0 con 0x7f73d80b9f80 2026-02-25T15:01:42.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.841+0000 7f73d4ff9640 10 client.4532.objecter ms_dispatch2 0x7f73d8000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.841+0000 7f73d4ff9640 3 client.4532.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.841+0000 7f73d4ff9640 3 client.4532.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.842+0000 7f73d4ff9640 20 client.4532.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1577485287 0 0) 0x7f2ca00b33e0 con 0x7f2ca00b9f80 2026-02-25T15:01:42.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 >> v1:10.20.193.93:6789/0 conn(0x7f2ca00585e0 legacy=0x7f2ca00b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 >> v1:10.20.193.90:6789/0 conn(0x7f2ca00ba350 legacy=0x7f2ca0161a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2c8e7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:01:42.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2c8e7fc640 1 monclient: found mon.c 2026-02-25T15:01:42.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2c8e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:42.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2c8e7fc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca00b36c0 con 0x7f2ca00b9f80 2026-02-25T15:01:42.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2c8e7fc640 10 monclient: _finish_auth 0 2026-02-25T15:01:42.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2c8e7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:01:42.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2c8e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:42.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2ca80aa640 5 monclient: authenticate success, global_id 4547 2026-02-25T15:01:42.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2ca80aa640 10 monclient: _renew_subs 2026-02-25T15:01:42.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2ca80aa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2ca80aa640 1 -- 10.20.193.90:0/1174542173 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca0161270 con 0x7f2ca00b9f80 2026-02-25T15:01:42.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2ca80aa640 10 client.4547.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:42.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2ca80aa640 10 monclient: _renew_subs 2026-02-25T15:01:42.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2ca80aa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2ca80aa640 1 -- 10.20.193.90:0/1174542173 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2ca0162180 con 0x7f2ca00b9f80 2026-02-25T15:01:42.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2ca80aa640 1 librados: init done 2026-02-25T15:01:42.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f73d4ff9640 1 -- 10.20.193.90:0/2213197521 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f73d8159080 con 0x7f73d80b9f80 2026-02-25T15:01:42.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f73d4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f73d4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:42.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2ca80aa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2ca80aa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2ca80aa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.844+0000 7f2ca80aa640 1 -- 10.20.193.90:0/1174542173 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca00b91d0 con 0x7f2ca00b9f80 2026-02-25T15:01:42.850 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.849+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fcc90003410 con 0x7fcc9c0585e0 2026-02-25T15:01:42.850 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.849+0000 7fcc82ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.850 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.849+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fcc900037a0 con 0x7fcc9c0585e0 2026-02-25T15:01:42.851 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.849+0000 7fcc82ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.851 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.849+0000 7fcc82ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.851 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.849+0000 7fcc82ffd640 10 monclient: dump: 2026-02-25T15:01:42.851 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.851 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.851 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.851 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.851 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.851 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.851 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.851 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.852 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.852 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.849+0000 7fcc98d72640 4 set_mon_vals no callback set 2026-02-25T15:01:42.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.850+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fcc90003980 con 0x7fcc9c0585e0 2026-02-25T15:01:42.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.850+0000 7fcc99d74640 10 monclient: get_auth_request con 0x7fcc68079fa0 auth_method 0 2026-02-25T15:01:42.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.851+0000 7fcc82ffd640 10 client.4529.objecter ms_handle_connect 0x7fcc68079fa0 2026-02-25T15:01:42.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.852+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7fcc90094940 con 0x7fcc9c0585e0 2026-02-25T15:01:42.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.852+0000 7fcc82ffd640 10 client.4529.objecter ms_dispatch2 0x7fcc9c000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.852+0000 7fcc82ffd640 3 client.4529.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.852+0000 7fcc82ffd640 3 client.4529.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.852+0000 7fcc82ffd640 20 client.4529.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.853+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fcc9c158b80 con 0x7fcc9c0585e0 2026-02-25T15:01:42.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.853+0000 7fcc82ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.853+0000 7fcc82ffd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:42.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.854+0000 7f5920ff9640 10 client.?.objecter ms_handle_connect 0x7f59240ba350 2026-02-25T15:01:42.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.854+0000 7f5920ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.862 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.861+0000 7f2547fff640 1 -- 10.20.193.90:0/359940460 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+12 (unknown 3207554136 0 778662779) 0x7f256405ee40 con 0x7f2570157780 2026-02-25T15:01:42.862 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.861+0000 7f2547fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-25T15:01:42.862 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.861+0000 7f2547fff640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:42.862 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:42.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.861+0000 7fe4fb7fe640 10 client.?.objecter ms_handle_connect 0x7fe51c0ba350 2026-02-25T15:01:42.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.861+0000 7fe4fb7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.864 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.862+0000 7f608affd640 10 client.?.objecter ms_handle_connect 0x7f60a40585e0 2026-02-25T15:01:42.864 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.862+0000 7f608affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.865 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.863+0000 7f257584d640 10 librados: watch_flush enter 2026-02-25T15:01:42.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.863+0000 7f257584d640 10 librados: watch_flush exit 2026-02-25T15:01:42.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.863+0000 7f257584d640 20 client.4505.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.863+0000 7f257584d640 10 client.4505.objecter successfully canceled tick 2026-02-25T15:01:42.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.864+0000 7f257584d640 1 -- 10.20.193.90:0/359940460 >> v1:10.20.193.189:6800/3647375982 conn(0x7f2540079f80 legacy=0x7f254009a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.864+0000 7f257584d640 10 monclient: shutdown 2026-02-25T15:01:42.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.864+0000 7f257584d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.864+0000 7f257584d640 1 -- 10.20.193.90:0/359940460 >> v1:10.20.193.94:6789/0 conn(0x7f2570157780 legacy=0x7f2570156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.864+0000 7f257584d640 1 -- 10.20.193.90:0/359940460 shutdown_connections 2026-02-25T15:01:42.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.864+0000 7f257584d640 1 -- 10.20.193.90:0/359940460 >> 10.20.193.90:0/359940460 conn(0x7f25700b2ef0 msgr2=0x7f25700b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.864+0000 7f257584d640 1 -- 10.20.193.90:0/359940460 shutdown_connections 2026-02-25T15:01:42.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.864+0000 7f257584d640 1 -- 10.20.193.90:0/359940460 wait complete. 2026-02-25T15:01:42.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.864+0000 7f257584d640 1 librados: shutdown 2026-02-25T15:01:42.874 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.8 2026-02-25T15:01:42.887 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.885+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f787c0048d0 con 0x7f78940b9f80 2026-02-25T15:01:42.887 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.885+0000 7f78897fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.887 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.885+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f787c004b40 con 0x7f78940b9f80 2026-02-25T15:01:42.887 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.885+0000 7f78897fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.887 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.885+0000 7f78897fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.887 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.885+0000 7f78897fa640 10 monclient: dump: 2026-02-25T15:01:42.887 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.887 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.888 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.888 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.888 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.888 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.888 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.888 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.888 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.888 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.888 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.885+0000 7f788affd640 4 set_mon_vals no callback set 2026-02-25T15:01:42.889 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.886+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f787c004d20 con 0x7f78940b9f80 2026-02-25T15:01:42.889 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.887+0000 7f7898eff640 10 monclient: get_auth_request con 0x7f7864079e60 auth_method 0 2026-02-25T15:01:42.889 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.887+0000 7f78897fa640 10 client.4530.objecter ms_handle_connect 0x7f7864079e60 2026-02-25T15:01:42.892 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.891+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f787c095c40 con 0x7f78940b9f80 2026-02-25T15:01:42.892 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.891+0000 7f78897fa640 10 client.4530.objecter ms_dispatch2 0x7f7894000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.892 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.891+0000 7f78897fa640 3 client.4530.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.893 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.891+0000 7f78897fa640 3 client.4530.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.893 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.891+0000 7f78897fa640 20 client.4530.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.893 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.891+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f787c05f5a0 con 0x7f78940b9f80 2026-02-25T15:01:42.893 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.891+0000 7f78897fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.893 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.891+0000 7f78897fa640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:42.893 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.892+0000 7f608affd640 10 client.?.objecter ms_handle_connect 0x7f60a40ba350 2026-02-25T15:01:42.893 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.892+0000 7f608affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:42.897 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.895+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0de0004c50 con 0x7f0df01fd410 2026-02-25T15:01:42.897 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.896+0000 7f0dcf7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.897 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.896+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0de0005060 con 0x7f0df01fd410 2026-02-25T15:01:42.897 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.896+0000 7f0dcf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.897 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.896+0000 7f0dcf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:42.897 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.896+0000 7f0dcf7fe640 10 monclient: dump: 2026-02-25T15:01:42.897 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.897 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.897 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.898 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.898 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.898 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.898 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.898 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.898 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.898 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.898 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.896+0000 7f0ded573640 4 set_mon_vals no callback set 2026-02-25T15:01:42.899 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.896+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f0de0005240 con 0x7f0df01fd410 2026-02-25T15:01:42.899 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.897+0000 7f0deed76640 10 monclient: get_auth_request con 0x7f0dbc079ef0 auth_method 0 2026-02-25T15:01:42.899 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.897+0000 7f0dcf7fe640 10 client.4527.objecter ms_handle_connect 0x7f0dbc079ef0 2026-02-25T15:01:42.901 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.900+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f0de0096030 con 0x7f0df01fd410 2026-02-25T15:01:42.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.900+0000 7f0dcf7fe640 10 client.4527.objecter ms_dispatch2 0x7f0df0000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.900+0000 7f0dcf7fe640 3 client.4527.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.900+0000 7f0dcf7fe640 3 client.4527.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.900+0000 7f0dcf7fe640 20 client.4527.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.900+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f0df00b2190 con 0x7f0df01fd410 2026-02-25T15:01:42.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.900+0000 7f0dcf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.900+0000 7f0dcf7fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:42.909 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.907+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2857865104 0 0) 0x7fe51c1fc940 con 0x7fe51c0585e0 2026-02-25T15:01:42.909 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.907+0000 7fe4fb7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:42.909 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.907+0000 7fe4fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:42.909 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.907+0000 7fe4fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:42.909 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.907+0000 7fe4fb7fe640 10 monclient(hunting): my global_id is 4557 2026-02-25T15:01:42.909 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.907+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4e8003610 con 0x7fe51c0585e0 2026-02-25T15:01:42.950 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.948+0000 7faffee7b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-25T15:01:42.950 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.948+0000 7faffee7b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-25T15:01:42.950 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.948+0000 7faffee7b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.950 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.948+0000 7faffee7b640 1 -- 10.20.193.90:0/4251103413 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7faff813a2b0 con 0x7faff80ba350 2026-02-25T15:01:42.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.956+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2c94003070 con 0x7f2ca00b9f80 2026-02-25T15:01:42.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.956+0000 7f2c8e7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:42.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.956+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2c94004d20 con 0x7f2ca00b9f80 2026-02-25T15:01:42.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.956+0000 7f2c8e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:42.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.956+0000 7f2c8e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:42.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.956+0000 7f2c8e7fc640 10 monclient: dump: 2026-02-25T15:01:42.958 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:42.958 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:42.958 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.958 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:42.958 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:42.958 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:42.958 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:42.959 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:42.959 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:42.959 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:42.959 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.956+0000 7f2c8ffff640 4 set_mon_vals no callback set 2026-02-25T15:01:42.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.957+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f2c94004f00 con 0x7f2ca00b9f80 2026-02-25T15:01:42.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.957+0000 7f2ca5e1f640 10 monclient: get_auth_request con 0x7f2c70079f30 auth_method 0 2026-02-25T15:01:42.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.958+0000 7f2c8e7fc640 10 client.4547.objecter ms_handle_connect 0x7f2c70079f30 2026-02-25T15:01:42.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.958+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f2c94095c40 con 0x7f2ca00b9f80 2026-02-25T15:01:42.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.958+0000 7f2c8e7fc640 10 client.4547.objecter ms_dispatch2 0x7f2ca0000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:42.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.958+0000 7f2c8e7fc640 3 client.4547.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:42.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.958+0000 7f2c8e7fc640 3 client.4547.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:42.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.958+0000 7f2c8e7fc640 20 client.4547.objecter dump_active .. 0 homeless 2026-02-25T15:01:42.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.959+0000 7f73de682640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-25T15:01:42.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.959+0000 7f73de682640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-25T15:01:42.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.959+0000 7f73de682640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.959+0000 7f73de682640 1 -- 10.20.193.90:0/2213197521 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f73d815b710 con 0x7f73d80b9f80 2026-02-25T15:01:42.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.960+0000 7fafed7fa640 1 -- 10.20.193.90:0/4251103413 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+12 (unknown 2894986287 0 778662779) 0x7faff405ded0 con 0x7faff80ba350 2026-02-25T15:01:42.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.960+0000 7fafed7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-25T15:01:42.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.960+0000 7fafed7fa640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:42.962 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:42.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.960+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f2ca0163930 con 0x7f2ca00b9f80 2026-02-25T15:01:42.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.960+0000 7f2c8e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:42.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.960+0000 7f2c8e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:42.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7faffee7b640 10 librados: watch_flush enter 2026-02-25T15:01:42.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7faffee7b640 10 librados: watch_flush exit 2026-02-25T15:01:42.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7faffee7b640 20 client.4523.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7faffee7b640 10 client.4523.objecter successfully canceled tick 2026-02-25T15:01:42.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7faffee7b640 1 -- 10.20.193.90:0/4251103413 >> v1:10.20.193.189:6800/3647375982 conn(0x7fafc8079ec0 legacy=0x7fafc809a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7faffee7b640 10 monclient: shutdown 2026-02-25T15:01:42.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7faffee7b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7faffee7b640 1 -- 10.20.193.90:0/4251103413 >> v1:10.20.193.94:6789/0 conn(0x7faff80ba350 legacy=0x7faff825f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7faffee7b640 1 -- 10.20.193.90:0/4251103413 shutdown_connections 2026-02-25T15:01:42.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7faffee7b640 1 -- 10.20.193.90:0/4251103413 >> 10.20.193.90:0/4251103413 conn(0x7faff80b7990 msgr2=0x7faff8056830 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.965 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7f73d4ff9640 1 -- 10.20.193.90:0/2213197521 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+12 (unknown 3646472635 0 778662779) 0x7f73c805e420 con 0x7f73d80b9f80 2026-02-25T15:01:42.965 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7f73d4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-25T15:01:42.965 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7f73d4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:42.965 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7faffee7b640 1 -- 10.20.193.90:0/4251103413 shutdown_connections 2026-02-25T15:01:42.966 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7faffee7b640 1 -- 10.20.193.90:0/4251103413 wait complete. 2026-02-25T15:01:42.966 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:42.966 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.962+0000 7faffee7b640 1 librados: shutdown 2026-02-25T15:01:42.967 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.965+0000 7f73de682640 10 librados: watch_flush enter 2026-02-25T15:01:42.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.965+0000 7f73de682640 10 librados: watch_flush exit 2026-02-25T15:01:42.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.965+0000 7f73de682640 20 client.4532.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.965+0000 7f73de682640 10 client.4532.objecter successfully canceled tick 2026-02-25T15:01:42.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.965+0000 7f73de682640 1 -- 10.20.193.90:0/2213197521 >> v1:10.20.193.189:6800/3647375982 conn(0x7f73a8079e50 legacy=0x7f73a809a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.965+0000 7f73de682640 10 monclient: shutdown 2026-02-25T15:01:42.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.965+0000 7f73de682640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.965+0000 7f73de682640 1 -- 10.20.193.90:0/2213197521 >> v1:10.20.193.94:6789/0 conn(0x7f73d80b9f80 legacy=0x7f73d8155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.965+0000 7f73de682640 1 -- 10.20.193.90:0/2213197521 shutdown_connections 2026-02-25T15:01:42.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.965+0000 7f73de682640 1 -- 10.20.193.90:0/2213197521 >> 10.20.193.90:0/2213197521 conn(0x7f73d80b7990 msgr2=0x7f73d80b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.965+0000 7f73de682640 1 -- 10.20.193.90:0/2213197521 shutdown_connections 2026-02-25T15:01:42.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.965+0000 7f73de682640 1 -- 10.20.193.90:0/2213197521 wait complete. 2026-02-25T15:01:42.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.965+0000 7f73de682640 1 librados: shutdown 2026-02-25T15:01:42.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.966+0000 7fcca0ee8640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-25T15:01:42.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.966+0000 7fcca0ee8640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-25T15:01:42.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.966+0000 7fcca0ee8640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:42.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.966+0000 7fcca0ee8640 1 -- 10.20.193.90:0/129879591 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 12} v 0) -- 0x7fcc9c0b6c80 con 0x7fcc9c0585e0 2026-02-25T15:01:42.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.968+0000 7fcc82ffd640 1 -- 10.20.193.90:0/129879591 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 12}]=0 v0) ==== 75+0+12 (unknown 4263375189 0 1257367751) 0x7fcc9005e2a0 con 0x7fcc9c0585e0 2026-02-25T15:01:42.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.968+0000 7fcc82ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-25T15:01:42.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.968+0000 7fcc82ffd640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:42.970 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:42.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.971+0000 7fcca0ee8640 10 librados: watch_flush enter 2026-02-25T15:01:42.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.971+0000 7fcca0ee8640 10 librados: watch_flush exit 2026-02-25T15:01:42.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.971+0000 7fcca0ee8640 20 client.4529.objecter shutdown clearing up homeless session... 2026-02-25T15:01:42.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.971+0000 7fcca0ee8640 10 client.4529.objecter successfully canceled tick 2026-02-25T15:01:42.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.971+0000 7fcca0ee8640 1 -- 10.20.193.90:0/129879591 >> v1:10.20.193.189:6800/3647375982 conn(0x7fcc68079fa0 legacy=0x7fcc6809a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.971+0000 7fcca0ee8640 10 monclient: shutdown 2026-02-25T15:01:42.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.971+0000 7fcca0ee8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:42.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.971+0000 7fcca0ee8640 1 -- 10.20.193.90:0/129879591 >> v1:10.20.193.94:6789/0 conn(0x7fcc9c0585e0 legacy=0x7fcc9c1fd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:42.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.971+0000 7fcca0ee8640 1 -- 10.20.193.90:0/129879591 shutdown_connections 2026-02-25T15:01:42.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.971+0000 7fcca0ee8640 1 -- 10.20.193.90:0/129879591 >> 10.20.193.90:0/129879591 conn(0x7fcc9c0b7990 msgr2=0x7fcc9c0b8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:42.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.971+0000 7fcca0ee8640 1 -- 10.20.193.90:0/129879591 shutdown_connections 2026-02-25T15:01:42.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.971+0000 7fcca0ee8640 1 -- 10.20.193.90:0/129879591 wait complete. 2026-02-25T15:01:42.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:42.971+0000 7fcca0ee8640 1 librados: shutdown 2026-02-25T15:01:42.975 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.9 2026-02-25T15:01:42.976 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.10 2026-02-25T15:01:42.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313220 for osd.12 2026-02-25T15:01:43.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.003+0000 7f789b18a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-25T15:01:43.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.003+0000 7f789b18a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-25T15:01:43.005 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.003+0000 7f789b18a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.005 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.003+0000 7f789b18a640 1 -- 10.20.193.90:0/3904297474 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 14} v 0) -- 0x7f789413a2b0 con 0x7f78940b9f80 2026-02-25T15:01:43.016 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.015+0000 7f0df5171640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-25T15:01:43.016 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.015+0000 7f0df5171640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-25T15:01:43.016 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.015+0000 7f0df5171640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.016 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.015+0000 7f0df5171640 1 -- 10.20.193.90:0/3673364815 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0df005bbc0 con 0x7f0df01fd410 2026-02-25T15:01:43.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.068+0000 7f2ca80aa640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-25T15:01:43.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.068+0000 7f2ca80aa640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-25T15:01:43.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.068+0000 7f2ca80aa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:43.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.068+0000 7f2ca80aa640 1 -- 10.20.193.90:0/1174542173 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2ca00b8ef0 con 0x7f2ca00b9f80 2026-02-25T15:01:43.072 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.070+0000 7f2c8e7fc640 1 -- 10.20.193.90:0/1174542173 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (unknown 1482059429 0 521551722) 0x7f2c9405f5a0 con 0x7f2ca00b9f80 2026-02-25T15:01:43.072 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.070+0000 7f2c8e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-25T15:01:43.072 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.070+0000 7f2c8e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:43.072 INFO:teuthology.orchestra.run.trial090.stdout:94489280516 2026-02-25T15:01:43.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.072+0000 7f2ca80aa640 10 librados: watch_flush enter 2026-02-25T15:01:43.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.072+0000 7f2ca80aa640 10 librados: watch_flush exit 2026-02-25T15:01:43.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.072+0000 7f2ca80aa640 20 client.4547.objecter shutdown clearing up homeless session... 2026-02-25T15:01:43.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.072+0000 7f2ca80aa640 10 client.4547.objecter successfully canceled tick 2026-02-25T15:01:43.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.072+0000 7f2ca80aa640 1 -- 10.20.193.90:0/1174542173 >> v1:10.20.193.189:6800/3647375982 conn(0x7f2c70079f30 legacy=0x7f2c7009a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.072+0000 7f2ca80aa640 10 monclient: shutdown 2026-02-25T15:01:43.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.072+0000 7f2ca80aa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:43.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.072+0000 7f2ca80aa640 1 -- 10.20.193.90:0/1174542173 >> v1:10.20.193.94:6789/0 conn(0x7f2ca00b9f80 legacy=0x7f2ca01fe4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.072+0000 7f2ca80aa640 1 -- 10.20.193.90:0/1174542173 shutdown_connections 2026-02-25T15:01:43.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.072+0000 7f2ca80aa640 1 -- 10.20.193.90:0/1174542173 >> 10.20.193.90:0/1174542173 conn(0x7f2ca00b7990 msgr2=0x7f2ca0158a10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:43.075 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.072+0000 7f2ca80aa640 1 -- 10.20.193.90:0/1174542173 shutdown_connections 2026-02-25T15:01:43.075 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.072+0000 7f2ca80aa640 1 -- 10.20.193.90:0/1174542173 wait complete. 2026-02-25T15:01:43.075 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.072+0000 7f2ca80aa640 1 librados: shutdown 2026-02-25T15:01:43.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280516 for osd.7 2026-02-25T15:01:43.089 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.088+0000 7f9103fff640 1 -- 10.20.193.90:0/433540121 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (unknown 1274345170 0 778662779) 0x7f911c05f0b0 con 0x7f912c162f10 2026-02-25T15:01:43.089 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.088+0000 7f9103fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-25T15:01:43.090 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.088+0000 7f9103fff640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:43.090 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:43.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.088+0000 7f5920ff9640 1 -- 10.20.193.90:0/1776190241 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 527349937 0 0) 0x7f5924162b20 con 0x7f59240585e0 2026-02-25T15:01:43.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.088+0000 7f5920ff9640 1 -- 10.20.193.90:0/1776190241 >> v1:10.20.193.94:6789/0 conn(0x7f59240ba350 legacy=0x7f5924157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.088+0000 7f5920ff9640 1 -- 10.20.193.90:0/1776190241 >> v1:10.20.193.93:6789/0 conn(0x7f59240b9f80 legacy=0x7f592415e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.088+0000 7f5920ff9640 10 monclient: _finish_hunting 0 2026-02-25T15:01:43.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.088+0000 7f5920ff9640 1 monclient: found mon.a 2026-02-25T15:01:43.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.088+0000 7f5920ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:43.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.088+0000 7f5920ff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.088+0000 7f5920ff9640 1 -- 10.20.193.90:0/1776190241 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59241662d0 con 0x7f59240585e0 2026-02-25T15:01:43.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.088+0000 7f5920ff9640 10 monclient: _finish_auth 0 2026-02-25T15:01:43.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.088+0000 7f5920ff9640 10 monclient: _check_auth_tickets 2026-02-25T15:01:43.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.088+0000 7f5920ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:43.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.088+0000 7f592a5a9640 5 monclient: authenticate success, global_id 4554 2026-02-25T15:01:43.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.089+0000 7f592a5a9640 10 monclient: _renew_subs 2026-02-25T15:01:43.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.089+0000 7f592a5a9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.089+0000 7f592a5a9640 1 -- 10.20.193.90:0/1776190241 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f59241665b0 con 0x7f59240585e0 2026-02-25T15:01:43.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.089+0000 7f592a5a9640 10 client.4554.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:43.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.089+0000 7f592a5a9640 10 monclient: _renew_subs 2026-02-25T15:01:43.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.089+0000 7f592a5a9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.089+0000 7f592a5a9640 1 -- 10.20.193.90:0/1776190241 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f59241fb230 con 0x7f59240585e0 2026-02-25T15:01:43.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.089+0000 7f592a5a9640 1 librados: init done 2026-02-25T15:01:43.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.089+0000 7f592a5a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.089+0000 7f592a5a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.089+0000 7f592a5a9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.089+0000 7f592a5a9640 1 -- 10.20.193.90:0/1776190241 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59240b9530 con 0x7f59240585e0 2026-02-25T15:01:43.095 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.091+0000 7f9131999640 10 librados: watch_flush enter 2026-02-25T15:01:43.095 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.091+0000 7f9131999640 10 librados: watch_flush exit 2026-02-25T15:01:43.095 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.091+0000 7f9131999640 20 client.4521.objecter shutdown clearing up homeless session... 2026-02-25T15:01:43.095 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.091+0000 7f9131999640 10 client.4521.objecter successfully canceled tick 2026-02-25T15:01:43.095 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.091+0000 7f9131999640 1 -- 10.20.193.90:0/433540121 >> v1:10.20.193.189:6800/3647375982 conn(0x7f90fc079f30 legacy=0x7f90fc09a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.095 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.091+0000 7f9131999640 10 monclient: shutdown 2026-02-25T15:01:43.096 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.091+0000 7f9131999640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:43.096 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.091+0000 7f9131999640 1 -- 10.20.193.90:0/433540121 >> v1:10.20.193.90:6789/0 conn(0x7f912c162f10 legacy=0x7f912c1f7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.096 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.091+0000 7f9131999640 1 -- 10.20.193.90:0/433540121 shutdown_connections 2026-02-25T15:01:43.096 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.091+0000 7f9131999640 1 -- 10.20.193.90:0/433540121 >> 10.20.193.90:0/433540121 conn(0x7f912c0b2ef0 msgr2=0x7f912c1603d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:43.096 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.091+0000 7f9131999640 1 -- 10.20.193.90:0/433540121 shutdown_connections 2026-02-25T15:01:43.096 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.091+0000 7f9131999640 1 -- 10.20.193.90:0/433540121 wait complete. 2026-02-25T15:01:43.096 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.091+0000 7f9131999640 1 librados: shutdown 2026-02-25T15:01:43.097 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.094+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f45fc0038d0 con 0x7f460c0ba350 2026-02-25T15:01:43.097 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.094+0000 7f45eb7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:43.097 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.094+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f45fc004df0 con 0x7f460c0ba350 2026-02-25T15:01:43.097 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.094+0000 7f45eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:43.097 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.094+0000 7f45eb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:43.097 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.094+0000 7f45eb7fe640 10 monclient: dump: 2026-02-25T15:01:43.098 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.098 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.098 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.098 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.098 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.098 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.098 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.098 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.098 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.098 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.098 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.094+0000 7f46097fa640 4 set_mon_vals no callback set 2026-02-25T15:01:43.098 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.095+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f45fc01e2d0 con 0x7f460c0ba350 2026-02-25T15:01:43.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.096+0000 7f460affd640 10 monclient: get_auth_request con 0x7f45d8079f10 auth_method 0 2026-02-25T15:01:43.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.096+0000 7f45eb7fe640 10 client.4542.objecter ms_handle_connect 0x7f45d8079f10 2026-02-25T15:01:43.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.098+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f45fc095e40 con 0x7f460c0ba350 2026-02-25T15:01:43.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.098+0000 7f45eb7fe640 10 client.4542.objecter ms_dispatch2 0x7f460c000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:43.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.099+0000 7f45eb7fe640 3 client.4542.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:43.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.099+0000 7f45eb7fe640 3 client.4542.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:43.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.099+0000 7f45eb7fe640 20 client.4542.objecter dump_active .. 0 homeless 2026-02-25T15:01:43.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.099+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f45fc05f7a0 con 0x7f460c0ba350 2026-02-25T15:01:43.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.099+0000 7f45eb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.099+0000 7f45eb7fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:43.102 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.6 2026-02-25T15:01:43.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.108+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f44bc002980 con 0x7f44d0059e70 2026-02-25T15:01:43.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.108+0000 7f44ccff9640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:43.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.108+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f44bc004b80 con 0x7f44d0059e70 2026-02-25T15:01:43.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.108+0000 7f44ccff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:43.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.108+0000 7f44ccff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:43.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.108+0000 7f44ccff9640 10 monclient: dump: 2026-02-25T15:01:43.110 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.110 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.110 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.110 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.110 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.111 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.111 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.111 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.111 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.111 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.108+0000 7f44ce7fc640 4 set_mon_vals no callback set 2026-02-25T15:01:43.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.108+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f44bc01e060 con 0x7f44d0059e70 2026-02-25T15:01:43.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.109+0000 7f44cf7fe640 10 monclient: get_auth_request con 0x7f44a0079e90 auth_method 0 2026-02-25T15:01:43.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.109+0000 7f44ccff9640 10 client.4545.objecter ms_handle_connect 0x7f44a0079e90 2026-02-25T15:01:43.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.111+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2916658252 0 0) 0x7f60a40af860 con 0x7f60a40ba350 2026-02-25T15:01:43.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.111+0000 7f608affd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.113 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.111+0000 7f608affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.113 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.111+0000 7f608affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.113 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.111+0000 7f608affd640 10 monclient(hunting): my global_id is 4560 2026-02-25T15:01:43.113 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.111+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6070003610 con 0x7f60a40ba350 2026-02-25T15:01:43.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.112+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f44bc095b70 con 0x7f44d0059e70 2026-02-25T15:01:43.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.112+0000 7f44ccff9640 10 client.4545.objecter ms_dispatch2 0x7f44d0000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:43.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.112+0000 7f44ccff9640 3 client.4545.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:43.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.112+0000 7f44ccff9640 3 client.4545.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:43.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.112+0000 7f44ccff9640 20 client.4545.objecter dump_active .. 0 homeless 2026-02-25T15:01:43.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.112+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f44d0162310 con 0x7f44d0059e70 2026-02-25T15:01:43.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.112+0000 7f44ccff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.112+0000 7f44ccff9640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:43.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.118+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4d38002dd0 con 0x7f4d48058070 2026-02-25T15:01:43.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.118+0000 7f4d35ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:43.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.118+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4d380031a0 con 0x7f4d48058070 2026-02-25T15:01:43.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.118+0000 7f4d35ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:43.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.118+0000 7f4d35ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:43.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.118+0000 7f4d35ffb640 10 monclient: dump: 2026-02-25T15:01:43.121 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.121 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.121 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.121 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.121 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.121 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.121 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.121 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.121 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.121 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.118+0000 7f4d377fe640 4 set_mon_vals no callback set 2026-02-25T15:01:43.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.119+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f4d38003380 con 0x7f4d48058070 2026-02-25T15:01:43.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.120+0000 7f4d4cc8e640 10 monclient: get_auth_request con 0x7f4d18079f20 auth_method 0 2026-02-25T15:01:43.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.120+0000 7f4d35ffb640 10 client.4551.objecter ms_handle_connect 0x7f4d18079f20 2026-02-25T15:01:43.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.122+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f4d38094a20 con 0x7f4d48058070 2026-02-25T15:01:43.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.122+0000 7f4d35ffb640 10 client.4551.objecter ms_dispatch2 0x7f4d48000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:43.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.122+0000 7f4d35ffb640 3 client.4551.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:43.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.122+0000 7f4d35ffb640 3 client.4551.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:43.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.122+0000 7f4d35ffb640 20 client.4551.objecter dump_active .. 0 homeless 2026-02-25T15:01:43.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.122+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f4d3805e380 con 0x7f4d48058070 2026-02-25T15:01:43.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.122+0000 7f4d35ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.122+0000 7f4d35ffb640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:43.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3032082614 0 0) 0x7fe51c1fcf00 con 0x7fe51c0ba350 2026-02-25T15:01:43.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7fe4fb7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7fe4fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7fe4fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7fe4fb7fe640 10 monclient(hunting): my global_id is 4559 2026-02-25T15:01:43.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe51c1fc940 con 0x7fe51c0ba350 2026-02-25T15:01:43.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4079600410 0 0) 0x7f60a40afe20 con 0x7f60a40585e0 2026-02-25T15:01:43.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7f608affd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7f608affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7f608affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7f608affd640 10 monclient(hunting): my global_id is 4562 2026-02-25T15:01:43.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60a40af860 con 0x7f60a40585e0 2026-02-25T15:01:43.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3239656941 0 0) 0x7fe4e8003610 con 0x7fe51c0585e0 2026-02-25T15:01:43.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe51c1fcf00 con 0x7fe51c0585e0 2026-02-25T15:01:43.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe510002870 con 0x7fe51c0585e0 2026-02-25T15:01:43.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7fe4fb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:43.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7fe4fb7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:43.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7fe4fb7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:43.170 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.170 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.170 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.170 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.170 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.170 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.170 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.170 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.170 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.170 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.165+0000 7fe4fb7fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:43.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.170+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3138917499 0 0) 0x7fe51c1fc940 con 0x7fe51c0ba350 2026-02-25T15:01:43.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.170+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe4e8003610 con 0x7fe51c0ba350 2026-02-25T15:01:43.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.170+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe50c003400 con 0x7fe51c0ba350 2026-02-25T15:01:43.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.170+0000 7fe4fb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:43.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.170+0000 7fe4fb7fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:43.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.170+0000 7fe4fb7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:43.172 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.172 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.172 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.172 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.172 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.172 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.173 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.173 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.173 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.173 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.170+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3601254410 0 0) 0x7f60a40af860 con 0x7f60a40585e0 2026-02-25T15:01:43.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.170+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f60a40afe20 con 0x7f60a40585e0 2026-02-25T15:01:43.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.170+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6098002870 con 0x7f60a40585e0 2026-02-25T15:01:43.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.170+0000 7f608affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:43.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.170+0000 7f608affd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:43.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.170+0000 7f608affd640 10 monclient(hunting): dump: 2026-02-25T15:01:43.174 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.174 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.175 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.175 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.175 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.175 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.175 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.175 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.175 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.175 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.170+0000 7f608affd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:43.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2769522497 0 0) 0x7fe4e8003610 con 0x7fe51c0ba350 2026-02-25T15:01:43.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 >> v1:10.20.193.93:6789/0 conn(0x7fe51c0b9f80 legacy=0x7fe51c1fdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 >> v1:10.20.193.90:6789/0 conn(0x7fe51c0585e0 legacy=0x7fe51c165500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe4fb7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:43.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe4fb7fe640 1 monclient: found mon.c 2026-02-25T15:01:43.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe4fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:43.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe4fb7fe640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:43.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c1fd1e0 con 0x7fe51c0ba350 2026-02-25T15:01:43.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe4fb7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:43.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe4fb7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:43.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe4fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:43.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe52125b640 5 monclient: authenticate success, global_id 4559 2026-02-25T15:01:43.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe52125b640 10 monclient: _renew_subs 2026-02-25T15:01:43.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe52125b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:43.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe52125b640 1 -- 10.20.193.90:0/4205025154 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe51c1fd4c0 con 0x7fe51c0ba350 2026-02-25T15:01:43.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe52125b640 10 client.4559.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:43.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe52125b640 10 monclient: _renew_subs 2026-02-25T15:01:43.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe52125b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:43.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe52125b640 1 -- 10.20.193.90:0/4205025154 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe51c163610 con 0x7fe51c0ba350 2026-02-25T15:01:43.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.172+0000 7fe52125b640 1 librados: init done 2026-02-25T15:01:43.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.173+0000 7fe52125b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.173+0000 7fe52125b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.173+0000 7fe52125b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:43.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.173+0000 7fe52125b640 1 -- 10.20.193.90:0/4205025154 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe51c0b7fa0 con 0x7fe51c0ba350 2026-02-25T15:01:43.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.173+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2935768460 0 0) 0x7f60a40afb40 con 0x7f60a40b9f80 2026-02-25T15:01:43.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.173+0000 7f608affd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.173+0000 7f608affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.173+0000 7f608affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.173+0000 7f608affd640 10 monclient(hunting): my global_id is 4537 2026-02-25T15:01:43.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.173+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60a40af860 con 0x7f60a40b9f80 2026-02-25T15:01:43.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3834996662 0 0) 0x7f60a40afe20 con 0x7f60a40585e0 2026-02-25T15:01:43.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 >> v1:10.20.193.93:6789/0 conn(0x7f60a40b9f80 legacy=0x7f60a41fc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 >> v1:10.20.193.90:6789/0 conn(0x7f60a40ba350 legacy=0x7f60a41656b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f608affd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:43.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f608affd640 1 monclient: found mon.c 2026-02-25T15:01:43.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f608affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:43.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f608affd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:43.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a40b0100 con 0x7f60a40585e0 2026-02-25T15:01:43.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f608affd640 10 monclient: _finish_auth 0 2026-02-25T15:01:43.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f608affd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:43.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f608affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:43.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f60a8dc8640 5 monclient: authenticate success, global_id 4562 2026-02-25T15:01:43.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f60a8dc8640 10 monclient: _renew_subs 2026-02-25T15:01:43.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f60a8dc8640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:43.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f60a8dc8640 1 -- 10.20.193.90:0/2902635105 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f60a41fc2d0 con 0x7f60a40585e0 2026-02-25T15:01:43.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f60a8dc8640 10 client.4562.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:43.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f60a8dc8640 10 monclient: _renew_subs 2026-02-25T15:01:43.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f60a8dc8640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:43.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f60a8dc8640 1 -- 10.20.193.90:0/2902635105 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f60a41fcf00 con 0x7f60a40585e0 2026-02-25T15:01:43.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.174+0000 7f60a8dc8640 1 librados: init done 2026-02-25T15:01:43.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.175+0000 7f60a8dc8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.175+0000 7f60a8dc8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.175+0000 7f60a8dc8640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:43.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.175+0000 7f60a8dc8640 1 -- 10.20.193.90:0/2902635105 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a40b7fa0 con 0x7f60a40585e0 2026-02-25T15:01:43.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.210+0000 7f461167c640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-25T15:01:43.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.210+0000 7f461167c640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-25T15:01:43.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.210+0000 7f461167c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.210+0000 7f461167c640 1 -- 10.20.193.90:0/2533946933 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f460c0b9580 con 0x7f460c0ba350 2026-02-25T15:01:43.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.219+0000 7f44d6604640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-25T15:01:43.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.219+0000 7f44d6604640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-25T15:01:43.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.219+0000 7f44d6604640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.221 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.219+0000 7f44d6604640 1 -- 10.20.193.90:0/458283867 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f44d0058590 con 0x7f44d0059e70 2026-02-25T15:01:43.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.235+0000 7f4d4f71a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-25T15:01:43.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.235+0000 7f4d4f71a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-25T15:01:43.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.235+0000 7f4d4f71a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.235+0000 7f4d4f71a640 1 -- 10.20.193.90:0/1258207199 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4d48161380 con 0x7f4d48058070 2026-02-25T15:01:43.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.266+0000 7f78897fa640 1 -- 10.20.193.90:0/3904297474 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 14}]=0 v0) ==== 75+0+12 (unknown 2547163239 0 778662779) 0x7f787c05eea0 con 0x7f78940b9f80 2026-02-25T15:01:43.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.266+0000 7f78897fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-25T15:01:43.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.266+0000 7f78897fa640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:43.268 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:43.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.268+0000 7f789b18a640 10 librados: watch_flush enter 2026-02-25T15:01:43.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.268+0000 7f789b18a640 10 librados: watch_flush exit 2026-02-25T15:01:43.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.268+0000 7f789b18a640 20 client.4530.objecter shutdown clearing up homeless session... 2026-02-25T15:01:43.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.268+0000 7f789b18a640 10 client.4530.objecter successfully canceled tick 2026-02-25T15:01:43.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.268+0000 7f789b18a640 1 -- 10.20.193.90:0/3904297474 >> v1:10.20.193.189:6800/3647375982 conn(0x7f7864079e60 legacy=0x7f786409a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.268+0000 7f789b18a640 10 monclient: shutdown 2026-02-25T15:01:43.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.268+0000 7f789b18a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:43.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.268+0000 7f789b18a640 1 -- 10.20.193.90:0/3904297474 >> v1:10.20.193.90:6789/0 conn(0x7f78940b9f80 legacy=0x7f78941656b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.268+0000 7f789b18a640 1 -- 10.20.193.90:0/3904297474 shutdown_connections 2026-02-25T15:01:43.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.268+0000 7f789b18a640 1 -- 10.20.193.90:0/3904297474 >> 10.20.193.90:0/3904297474 conn(0x7f78940b7990 msgr2=0x7f7894056830 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:43.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.268+0000 7f789b18a640 1 -- 10.20.193.90:0/3904297474 shutdown_connections 2026-02-25T15:01:43.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.268+0000 7f789b18a640 1 -- 10.20.193.90:0/3904297474 wait complete. 2026-02-25T15:01:43.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.268+0000 7f789b18a640 1 librados: shutdown 2026-02-25T15:01:43.278 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.14 2026-02-25T15:01:43.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.280+0000 7f5920ff9640 1 -- 10.20.193.90:0/1776190241 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5910003380 con 0x7f59240585e0 2026-02-25T15:01:43.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.280+0000 7f5920ff9640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:43.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.280+0000 7f5920ff9640 1 -- 10.20.193.90:0/1776190241 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5910004c40 con 0x7f59240585e0 2026-02-25T15:01:43.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.280+0000 7f5920ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:43.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.280+0000 7f5920ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:43.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.280+0000 7f5920ff9640 10 monclient: dump: 2026-02-25T15:01:43.282 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.282 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.282 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.283 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.283 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.283 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.283 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.283 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.283 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.283 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.280+0000 7f59227fc640 4 set_mon_vals no callback set 2026-02-25T15:01:43.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.280+0000 7f0dcf7fe640 1 -- 10.20.193.90:0/3673364815 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (unknown 383520633 0 737253363) 0x7f0de005f990 con 0x7f0df01fd410 2026-02-25T15:01:43.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.280+0000 7f0dcf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-25T15:01:43.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.280+0000 7f0dcf7fe640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:43.284 INFO:teuthology.orchestra.run.trial090.stdout:94489280515 2026-02-25T15:01:43.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.281+0000 7f5920ff9640 1 -- 10.20.193.90:0/1776190241 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f591001e120 con 0x7f59240585e0 2026-02-25T15:01:43.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.281+0000 7f59237fe640 10 monclient: get_auth_request con 0x7f58f4079f60 auth_method 0 2026-02-25T15:01:43.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.282+0000 7f5920ff9640 10 client.4554.objecter ms_handle_connect 0x7f58f4079f60 2026-02-25T15:01:43.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.283+0000 7f0df5171640 10 librados: watch_flush enter 2026-02-25T15:01:43.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.283+0000 7f0df5171640 10 librados: watch_flush exit 2026-02-25T15:01:43.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.283+0000 7f0df5171640 20 client.4527.objecter shutdown clearing up homeless session... 2026-02-25T15:01:43.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.283+0000 7f0df5171640 10 client.4527.objecter successfully canceled tick 2026-02-25T15:01:43.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.283+0000 7f0df5171640 1 -- 10.20.193.90:0/3673364815 >> v1:10.20.193.189:6800/3647375982 conn(0x7f0dbc079ef0 legacy=0x7f0dbc09a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.283+0000 7f0df5171640 10 monclient: shutdown 2026-02-25T15:01:43.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.283+0000 7f0df5171640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:43.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.283+0000 7f0df5171640 1 -- 10.20.193.90:0/3673364815 >> v1:10.20.193.90:6789/0 conn(0x7f0df01fd410 legacy=0x7f0df0154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.283+0000 7f0df5171640 1 -- 10.20.193.90:0/3673364815 shutdown_connections 2026-02-25T15:01:43.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.283+0000 7f0df5171640 1 -- 10.20.193.90:0/3673364815 >> 10.20.193.90:0/3673364815 conn(0x7f0df00b5290 msgr2=0x7f0df00b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:43.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.283+0000 7f0df5171640 1 -- 10.20.193.90:0/3673364815 shutdown_connections 2026-02-25T15:01:43.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.283+0000 7f0df5171640 1 -- 10.20.193.90:0/3673364815 wait complete. 2026-02-25T15:01:43.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.283+0000 7f0df5171640 1 librados: shutdown 2026-02-25T15:01:43.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.286+0000 7f5920ff9640 1 -- 10.20.193.90:0/1776190241 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f5910095cf0 con 0x7f59240585e0 2026-02-25T15:01:43.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.286+0000 7f5920ff9640 10 client.4554.objecter ms_dispatch2 0x7f5924000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:43.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.286+0000 7f5920ff9640 3 client.4554.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:43.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.286+0000 7f5920ff9640 3 client.4554.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:43.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.286+0000 7f5920ff9640 20 client.4554.objecter dump_active .. 0 homeless 2026-02-25T15:01:43.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.286+0000 7f5920ff9640 1 -- 10.20.193.90:0/1776190241 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f591005f650 con 0x7f59240585e0 2026-02-25T15:01:43.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.286+0000 7f5920ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.286+0000 7f5920ff9640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:43.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280515 for osd.3 2026-02-25T15:01:43.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.295+0000 7f44ccff9640 1 -- 10.20.193.90:0/458283867 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+12 (unknown 3405404620 0 778662779) 0x7f44d0058590 con 0x7f44d0059e70 2026-02-25T15:01:43.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.295+0000 7f44ccff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-25T15:01:43.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.295+0000 7f44ccff9640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:43.297 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:43.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.297+0000 7f44d6604640 10 librados: watch_flush enter 2026-02-25T15:01:43.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.297+0000 7f44d6604640 10 librados: watch_flush exit 2026-02-25T15:01:43.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.297+0000 7f44d6604640 20 client.4545.objecter shutdown clearing up homeless session... 2026-02-25T15:01:43.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.297+0000 7f44d6604640 10 client.4545.objecter successfully canceled tick 2026-02-25T15:01:43.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.297+0000 7f44d6604640 1 -- 10.20.193.90:0/458283867 >> v1:10.20.193.189:6800/3647375982 conn(0x7f44a0079e90 legacy=0x7f44a009a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.297+0000 7f44d6604640 10 monclient: shutdown 2026-02-25T15:01:43.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.297+0000 7f44d6604640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:43.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.297+0000 7f44d6604640 1 -- 10.20.193.90:0/458283867 >> v1:10.20.193.90:6789/0 conn(0x7f44d0059e70 legacy=0x7f44d0056e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.298+0000 7f44d6604640 1 -- 10.20.193.90:0/458283867 shutdown_connections 2026-02-25T15:01:43.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.298+0000 7f44d6604640 1 -- 10.20.193.90:0/458283867 >> 10.20.193.90:0/458283867 conn(0x7f44d00590f0 msgr2=0x7f44d0132150 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:43.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.298+0000 7f44d6604640 1 -- 10.20.193.90:0/458283867 shutdown_connections 2026-02-25T15:01:43.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.298+0000 7f44d6604640 1 -- 10.20.193.90:0/458283867 wait complete. 2026-02-25T15:01:43.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.298+0000 7f44d6604640 1 librados: shutdown 2026-02-25T15:01:43.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.299+0000 7f45eb7fe640 1 -- 10.20.193.90:0/2533946933 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (unknown 92182286 0 778662779) 0x7f45fc05f0a0 con 0x7f460c0ba350 2026-02-25T15:01:43.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.299+0000 7f45eb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-25T15:01:43.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.299+0000 7f45eb7fe640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:43.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.299+0000 7f4d35ffb640 1 -- 10.20.193.90:0/1258207199 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (unknown 2131975755 0 778662779) 0x7f4d3805dc80 con 0x7f4d48058070 2026-02-25T15:01:43.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.299+0000 7f4d35ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-25T15:01:43.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.299+0000 7f4d35ffb640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:43.302 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:43.303 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:43.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.301+0000 7f461167c640 10 librados: watch_flush enter 2026-02-25T15:01:43.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.301+0000 7f461167c640 10 librados: watch_flush exit 2026-02-25T15:01:43.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.301+0000 7f461167c640 20 client.4542.objecter shutdown clearing up homeless session... 2026-02-25T15:01:43.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.301+0000 7f461167c640 10 client.4542.objecter successfully canceled tick 2026-02-25T15:01:43.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.301+0000 7f461167c640 1 -- 10.20.193.90:0/2533946933 >> v1:10.20.193.189:6800/3647375982 conn(0x7f45d8079f10 legacy=0x7f45d809a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.301+0000 7f461167c640 10 monclient: shutdown 2026-02-25T15:01:43.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.301+0000 7f461167c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:43.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.301+0000 7f461167c640 1 -- 10.20.193.90:0/2533946933 >> v1:10.20.193.90:6789/0 conn(0x7f460c0ba350 legacy=0x7f460c155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.301+0000 7f461167c640 1 -- 10.20.193.90:0/2533946933 shutdown_connections 2026-02-25T15:01:43.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.301+0000 7f461167c640 1 -- 10.20.193.90:0/2533946933 >> 10.20.193.90:0/2533946933 conn(0x7f460c0b7990 msgr2=0x7f460c159790 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:43.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f461167c640 1 -- 10.20.193.90:0/2533946933 shutdown_connections 2026-02-25T15:01:43.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f461167c640 1 -- 10.20.193.90:0/2533946933 wait complete. 2026-02-25T15:01:43.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f4d4f71a640 10 librados: watch_flush enter 2026-02-25T15:01:43.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f4d4f71a640 10 librados: watch_flush exit 2026-02-25T15:01:43.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f461167c640 1 librados: shutdown 2026-02-25T15:01:43.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f4d4f71a640 20 client.4551.objecter shutdown clearing up homeless session... 2026-02-25T15:01:43.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f4d4f71a640 10 client.4551.objecter successfully canceled tick 2026-02-25T15:01:43.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f4d4f71a640 1 -- 10.20.193.90:0/1258207199 >> v1:10.20.193.189:6800/3647375982 conn(0x7f4d18079f20 legacy=0x7f4d1809a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f4d4f71a640 10 monclient: shutdown 2026-02-25T15:01:43.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f4d4f71a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:43.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f4d4f71a640 1 -- 10.20.193.90:0/1258207199 >> v1:10.20.193.90:6789/0 conn(0x7f4d48058070 legacy=0x7f4d4813af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f4d4f71a640 1 -- 10.20.193.90:0/1258207199 shutdown_connections 2026-02-25T15:01:43.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f4d4f71a640 1 -- 10.20.193.90:0/1258207199 >> 10.20.193.90:0/1258207199 conn(0x7f4d480b7780 msgr2=0x7f4d481582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:43.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f4d4f71a640 1 -- 10.20.193.90:0/1258207199 shutdown_connections 2026-02-25T15:01:43.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f4d4f71a640 1 -- 10.20.193.90:0/1258207199 wait complete. 2026-02-25T15:01:43.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.302+0000 7f4d4f71a640 1 librados: shutdown 2026-02-25T15:01:43.309 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.11 2026-02-25T15:01:43.312 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.5 2026-02-25T15:01:43.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.2 2026-02-25T15:01:43.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.350+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe50c003a80 con 0x7fe51c0ba350 2026-02-25T15:01:43.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.350+0000 7fe4fb7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:43.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.350+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe50c003dd0 con 0x7fe51c0ba350 2026-02-25T15:01:43.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.350+0000 7fe4fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:43.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.350+0000 7fe4fb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:43.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.350+0000 7fe4fb7fe640 10 monclient: dump: 2026-02-25T15:01:43.353 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.353 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.353 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.353 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.353 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.353 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.353 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.353 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.353 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.353 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.350+0000 7fe519573640 4 set_mon_vals no callback set 2026-02-25T15:01:43.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.351+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fe50c01d2b0 con 0x7fe51c0ba350 2026-02-25T15:01:43.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.352+0000 7fe51ad76640 10 monclient: get_auth_request con 0x7fe4e8079f30 auth_method 0 2026-02-25T15:01:43.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.352+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7fe50c094d40 con 0x7fe51c0ba350 2026-02-25T15:01:43.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.352+0000 7fe4fb7fe640 10 client.4559.objecter ms_dispatch2 0x7fe51c000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:43.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.352+0000 7fe4fb7fe640 3 client.4559.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:43.356 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.352+0000 7fe4fb7fe640 3 client.4559.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:43.356 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.352+0000 7fe4fb7fe640 20 client.4559.objecter dump_active .. 0 homeless 2026-02-25T15:01:43.356 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.352+0000 7fe4fb7fe640 10 client.4559.objecter ms_handle_connect 0x7fe4e8079f30 2026-02-25T15:01:43.356 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.353+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fe51c0b7fa0 con 0x7fe51c0ba350 2026-02-25T15:01:43.356 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.353+0000 7fe4fb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.356 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.353+0000 7fe4fb7fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:43.361 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.359+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6098002bf0 con 0x7f60a40585e0 2026-02-25T15:01:43.361 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.359+0000 7f608affd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:43.361 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.359+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6098004c40 con 0x7f60a40585e0 2026-02-25T15:01:43.361 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.359+0000 7f608affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:43.361 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.359+0000 7f608affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:43.361 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.359+0000 7f608affd640 10 monclient: dump: 2026-02-25T15:01:43.361 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.361 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.362 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.362 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.362 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.362 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.362 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.362 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.362 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.362 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.362 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.359+0000 7f60a0d72640 4 set_mon_vals no callback set 2026-02-25T15:01:43.363 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.360+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f609801e120 con 0x7f60a40585e0 2026-02-25T15:01:43.364 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.361+0000 7f60a1d74640 10 monclient: get_auth_request con 0x7f6070079fd0 auth_method 0 2026-02-25T15:01:43.364 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.361+0000 7f608affd640 10 client.4562.objecter ms_handle_connect 0x7f6070079fd0 2026-02-25T15:01:43.364 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.361+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f6098095b50 con 0x7f60a40585e0 2026-02-25T15:01:43.364 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.361+0000 7f608affd640 10 client.4562.objecter ms_dispatch2 0x7f60a4000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:43.364 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.361+0000 7f608affd640 3 client.4562.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:43.364 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.361+0000 7f608affd640 3 client.4562.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:43.364 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.361+0000 7f608affd640 20 client.4562.objecter dump_active .. 0 homeless 2026-02-25T15:01:43.365 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.363+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f60a40b7fa0 con 0x7f60a40585e0 2026-02-25T15:01:43.365 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.363+0000 7f608affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.365 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.363+0000 7f608affd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:43.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.397+0000 7f592a5a9640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-25T15:01:43.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.397+0000 7f592a5a9640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-25T15:01:43.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.397+0000 7f592a5a9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.397+0000 7f592a5a9640 1 -- 10.20.193.90:0/1776190241 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 13} v 0) -- 0x7f59240b6480 con 0x7f59240585e0 2026-02-25T15:01:43.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.400+0000 7f5920ff9640 1 -- 10.20.193.90:0/1776190241 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 13}]=0 v0) ==== 75+0+12 (unknown 3988744482 0 737253363) 0x7f591005ef50 con 0x7f59240585e0 2026-02-25T15:01:43.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.400+0000 7f5920ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-25T15:01:43.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.400+0000 7f5920ff9640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:43.402 INFO:teuthology.orchestra.run.trial090.stdout:94489280515 2026-02-25T15:01:43.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.402+0000 7f592a5a9640 10 librados: watch_flush enter 2026-02-25T15:01:43.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.402+0000 7f592a5a9640 10 librados: watch_flush exit 2026-02-25T15:01:43.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.402+0000 7f592a5a9640 20 client.4554.objecter shutdown clearing up homeless session... 2026-02-25T15:01:43.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.402+0000 7f592a5a9640 10 client.4554.objecter successfully canceled tick 2026-02-25T15:01:43.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.402+0000 7f592a5a9640 1 -- 10.20.193.90:0/1776190241 >> v1:10.20.193.189:6800/3647375982 conn(0x7f58f4079f60 legacy=0x7f58f409a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.402+0000 7f592a5a9640 10 monclient: shutdown 2026-02-25T15:01:43.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.402+0000 7f592a5a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:43.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.402+0000 7f592a5a9640 1 -- 10.20.193.90:0/1776190241 >> v1:10.20.193.90:6789/0 conn(0x7f59240585e0 legacy=0x7f5924162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.402+0000 7f592a5a9640 1 -- 10.20.193.90:0/1776190241 shutdown_connections 2026-02-25T15:01:43.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.402+0000 7f592a5a9640 1 -- 10.20.193.90:0/1776190241 >> 10.20.193.90:0/1776190241 conn(0x7f59240b7990 msgr2=0x7f59240b5370 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:43.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.402+0000 7f592a5a9640 1 -- 10.20.193.90:0/1776190241 shutdown_connections 2026-02-25T15:01:43.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.402+0000 7f592a5a9640 1 -- 10.20.193.90:0/1776190241 wait complete. 2026-02-25T15:01:43.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.402+0000 7f592a5a9640 1 librados: shutdown 2026-02-25T15:01:43.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280515 for osd.13 2026-02-25T15:01:43.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.460+0000 7fe52125b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-25T15:01:43.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.460+0000 7fe52125b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-25T15:01:43.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.460+0000 7fe52125b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:43.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.460+0000 7fe52125b640 1 -- 10.20.193.90:0/4205025154 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 15} v 0) -- 0x7fe51c04b700 con 0x7fe51c0ba350 2026-02-25T15:01:43.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.462+0000 7fe4fb7fe640 1 -- 10.20.193.90:0/4205025154 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 15}]=0 v0) ==== 75+0+12 (unknown 2221946896 0 778662779) 0x7fe50c05e6a0 con 0x7fe51c0ba350 2026-02-25T15:01:43.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.462+0000 7fe4fb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-25T15:01:43.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.462+0000 7fe4fb7fe640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:43.464 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:43.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.464+0000 7fe52125b640 10 librados: watch_flush enter 2026-02-25T15:01:43.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.464+0000 7fe52125b640 10 librados: watch_flush exit 2026-02-25T15:01:43.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.464+0000 7fe52125b640 20 client.4559.objecter shutdown clearing up homeless session... 2026-02-25T15:01:43.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.464+0000 7fe52125b640 10 client.4559.objecter successfully canceled tick 2026-02-25T15:01:43.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.464+0000 7fe52125b640 1 -- 10.20.193.90:0/4205025154 >> v1:10.20.193.189:6800/3647375982 conn(0x7fe4e8079f30 legacy=0x7fe4e809a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.464+0000 7fe52125b640 10 monclient: shutdown 2026-02-25T15:01:43.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.464+0000 7fe52125b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:43.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.464+0000 7fe52125b640 1 -- 10.20.193.90:0/4205025154 >> v1:10.20.193.94:6789/0 conn(0x7fe51c0ba350 legacy=0x7fe51c155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.464+0000 7fe52125b640 1 -- 10.20.193.90:0/4205025154 shutdown_connections 2026-02-25T15:01:43.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.464+0000 7fe52125b640 1 -- 10.20.193.90:0/4205025154 >> 10.20.193.90:0/4205025154 conn(0x7fe51c0b7990 msgr2=0x7fe51c15bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:43.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.464+0000 7fe52125b640 1 -- 10.20.193.90:0/4205025154 shutdown_connections 2026-02-25T15:01:43.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.464+0000 7fe52125b640 1 -- 10.20.193.90:0/4205025154 wait complete. 2026-02-25T15:01:43.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.464+0000 7fe52125b640 1 librados: shutdown 2026-02-25T15:01:43.474 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.15 2026-02-25T15:01:43.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.477+0000 7f60a8dc8640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-25T15:01:43.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.477+0000 7f60a8dc8640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-25T15:01:43.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.477+0000 7f60a8dc8640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:43.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.477+0000 7f60a8dc8640 1 -- 10.20.193.90:0/2902635105 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f60a40b8ce0 con 0x7f60a40585e0 2026-02-25T15:01:43.481 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.480+0000 7f608affd640 1 -- 10.20.193.90:0/2902635105 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (unknown 1823589948 0 778662779) 0x7f60a4163930 con 0x7f60a40585e0 2026-02-25T15:01:43.481 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.480+0000 7f608affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-25T15:01:43.481 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.480+0000 7f608affd640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:43.481 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:43.484 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.482+0000 7f60a8dc8640 10 librados: watch_flush enter 2026-02-25T15:01:43.484 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.482+0000 7f60a8dc8640 10 librados: watch_flush exit 2026-02-25T15:01:43.484 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.482+0000 7f60a8dc8640 20 client.4562.objecter shutdown clearing up homeless session... 2026-02-25T15:01:43.484 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.482+0000 7f60a8dc8640 10 client.4562.objecter successfully canceled tick 2026-02-25T15:01:43.484 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.482+0000 7f60a8dc8640 1 -- 10.20.193.90:0/2902635105 >> v1:10.20.193.189:6800/3647375982 conn(0x7f6070079fd0 legacy=0x7f607009a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.485 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.482+0000 7f60a8dc8640 10 monclient: shutdown 2026-02-25T15:01:43.485 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.482+0000 7f60a8dc8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:43.485 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.482+0000 7f60a8dc8640 1 -- 10.20.193.90:0/2902635105 >> v1:10.20.193.94:6789/0 conn(0x7f60a40585e0 legacy=0x7f60a41564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.485 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.482+0000 7f60a8dc8640 1 -- 10.20.193.90:0/2902635105 shutdown_connections 2026-02-25T15:01:43.485 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.482+0000 7f60a8dc8640 1 -- 10.20.193.90:0/2902635105 >> 10.20.193.90:0/2902635105 conn(0x7f60a40b7990 msgr2=0x7f60a4157b30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:43.485 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.482+0000 7f60a8dc8640 1 -- 10.20.193.90:0/2902635105 shutdown_connections 2026-02-25T15:01:43.485 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.482+0000 7f60a8dc8640 1 -- 10.20.193.90:0/2902635105 wait complete. 2026-02-25T15:01:43.485 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.482+0000 7f60a8dc8640 1 librados: shutdown 2026-02-25T15:01:43.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.4 2026-02-25T15:01:43.742 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-25T15:01:43.748 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-25T15:01:43.809 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.808+0000 7f2968771640 10 monclient: get_monmap_and_config 2026-02-25T15:01:43.810 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.808+0000 7f2968771640 10 monclient: build_initial_monmap 2026-02-25T15:01:43.810 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.808+0000 7f2968771640 10 monclient: monmap: 2026-02-25T15:01:43.810 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:43.810 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.810 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:43.809519+0000 2026-02-25T15:01:43.810 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:43.809519+0000 2026-02-25T15:01:43.810 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:43.810 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.810 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:43.810 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:43.810 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:43.810 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.810 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.809+0000 7f2968771640 1 Processor -- start 2026-02-25T15:01:43.810 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.809+0000 7f2968771640 1 -- start start 2026-02-25T15:01:43.810 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.809+0000 7f2968771640 10 monclient: init 2026-02-25T15:01:43.811 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.810+0000 7f2968771640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:43.811 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.810+0000 7f2968771640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:43.811 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.810+0000 7f2968771640 10 monclient(hunting): picked mon.noname-b con 0x7f2960157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:43.811 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.810+0000 7f2968771640 10 monclient(hunting): picked mon.noname-c con 0x7f2960157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:43.811 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.810+0000 7f2968771640 10 monclient(hunting): picked mon.noname-a con 0x7f2960164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:43.811 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.810+0000 7f2968771640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f296005de70 con 0x7f2960164d00 2026-02-25T15:01:43.811 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.810+0000 7f2968771640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2960055b20 con 0x7f2960157780 2026-02-25T15:01:43.811 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.810+0000 7f2968771640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2960055e00 con 0x7f2960157b50 2026-02-25T15:01:43.811 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.810+0000 7f2968771640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:43.811 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.810+0000 7f2968771640 10 monclient(hunting): authenticate will time out at 2993.410363s 2026-02-25T15:01:43.811 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.810+0000 7f29664e6640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f2960157780 0x7f2960056ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:50978/0 (socket says 10.20.193.90:50978) 2026-02-25T15:01:43.811 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.810+0000 7f29664e6640 1 -- 10.20.193.90:0/1898297125 learned_addr learned my addr 10.20.193.90:0/1898297125 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:43.812 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.811+0000 7f29654e4640 1 -- 10.20.193.90:0/1898297125 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 312005347 0 0) 0x7f2960055b20 con 0x7f2960157780 2026-02-25T15:01:43.812 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.811+0000 7f29654e4640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.812 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.811+0000 7f29654e4640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.811+0000 7f29654e4640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.811+0000 7f29654e4640 10 monclient(hunting): my global_id is 4540 2026-02-25T15:01:43.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.811+0000 7f29654e4640 1 -- 10.20.193.90:0/1898297125 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2948003610 con 0x7f2960157780 2026-02-25T15:01:43.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.812+0000 7f29654e4640 1 -- 10.20.193.90:0/1898297125 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2526708744 0 0) 0x7f2960055e00 con 0x7f2960157b50 2026-02-25T15:01:43.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.812+0000 7f29654e4640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.812+0000 7f29654e4640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.812+0000 7f29654e4640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.812+0000 7f29654e4640 10 monclient(hunting): my global_id is 4565 2026-02-25T15:01:43.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.812+0000 7f29654e4640 1 -- 10.20.193.90:0/1898297125 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2960055b20 con 0x7f2960157b50 2026-02-25T15:01:43.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.812+0000 7f29654e4640 1 -- 10.20.193.90:0/1898297125 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 396750154 0 0) 0x7f296005de70 con 0x7f2960164d00 2026-02-25T15:01:43.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.812+0000 7f29654e4640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.812+0000 7f29654e4640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.812+0000 7f29654e4640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.812+0000 7f29654e4640 10 monclient(hunting): my global_id is 4563 2026-02-25T15:01:43.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.812+0000 7f29654e4640 1 -- 10.20.193.90:0/1898297125 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2960055e00 con 0x7f2960164d00 2026-02-25T15:01:43.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f29654e4640 1 -- 10.20.193.90:0/1898297125 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 126199610 0 0) 0x7f2948003610 con 0x7f2960157780 2026-02-25T15:01:43.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f29654e4640 1 -- 10.20.193.90:0/1898297125 >> v1:10.20.193.94:6789/0 conn(0x7f2960157b50 legacy=0x7f29601645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f29654e4640 1 -- 10.20.193.90:0/1898297125 >> v1:10.20.193.90:6789/0 conn(0x7f2960164d00 legacy=0x7f29601c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f29654e4640 10 monclient: _finish_hunting 0 2026-02-25T15:01:43.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f29654e4640 1 monclient: found mon.noname-b 2026-02-25T15:01:43.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f29654e4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:43.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f29654e4640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:43.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f29654e4640 1 -- 10.20.193.90:0/1898297125 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29600560e0 con 0x7f2960157780 2026-02-25T15:01:43.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f29654e4640 10 monclient: _finish_auth 0 2026-02-25T15:01:43.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f29654e4640 10 monclient: _check_auth_tickets 2026-02-25T15:01:43.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f29654e4640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:43.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f29654e4640 1 -- 10.20.193.90:0/1898297125 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2950002df0 con 0x7f2960157780 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f29654e4640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f29654e4640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f29654e4640 10 monclient: dump: 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f2968771640 5 monclient: authenticate success, global_id 4540 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.813+0000 7f2968771640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.814+0000 7f29654e4640 1 -- 10.20.193.90:0/1898297125 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2950002fc0 con 0x7f2960157780 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.814+0000 7f29654e4640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.814+0000 7f29654e4640 1 -- 10.20.193.90:0/1898297125 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2950003400 con 0x7f2960157780 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.814+0000 7f29654e4640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.814+0000 7f29654e4640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.814+0000 7f29654e4640 10 monclient: dump: 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.814+0000 7f2968771640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:43.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.814+0000 7f2968771640 4 set_mon_vals no callback set 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.814+0000 7f2968771640 10 monclient: shutdown 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.814+0000 7f2968771640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.814+0000 7f2968771640 1 -- 10.20.193.90:0/1898297125 >> v1:10.20.193.93:6789/0 conn(0x7f2960157780 legacy=0x7f2960056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.814+0000 7f2968771640 1 -- 10.20.193.90:0/1898297125 shutdown_connections 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.814+0000 7f2968771640 1 -- 10.20.193.90:0/1898297125 >> 10.20.193.90:0/1898297125 conn(0x7f29600b2ef0 msgr2=0x7f29600b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.814+0000 7f2968771640 1 -- 10.20.193.90:0/1898297125 shutdown_connections 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.815+0000 7f2968771640 1 -- 10.20.193.90:0/1898297125 wait complete. 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.815+0000 7f2968771640 10 monclient: build_initial_monmap 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.815+0000 7f2968771640 10 monclient: monmap: 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.815+0000 7f2968771640 1 librados: starting msgr at 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.815+0000 7f2968771640 1 librados: starting objecter 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.815+0000 7f2968771640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:43.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.815+0000 7f2968771640 1 Processor -- start 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.815+0000 7f2968771640 1 -- start start 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.815+0000 7f2968771640 1 librados: setting wanted keys 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.815+0000 7f2968771640 1 librados: calling monclient init 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.815+0000 7f2968771640 10 monclient: init 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2968771640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2968771640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2968771640 10 monclient(hunting): picked mon.noname-b con 0x7f2960157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2968771640 10 monclient(hunting): picked mon.noname-c con 0x7f2960157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2968771640 10 monclient(hunting): picked mon.noname-a con 0x7f2960164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2968771640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2960133250 con 0x7f2960164d00 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2968771640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2960133530 con 0x7f2960157780 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2968771640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2960133810 con 0x7f2960157b50 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2968771640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2968771640 10 monclient(hunting): authenticate will time out at 2993.416171s 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2966ce7640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f2960164d00 0x7f2960139f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57174/0 (socket says 10.20.193.90:57174) 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2966ce7640 1 -- 10.20.193.90:0/3056836534 learned_addr learned my addr 10.20.193.90:0/3056836534 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2946ffd640 10 client.?.objecter ms_handle_connect 0x7f2960164d00 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2946ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2946ffd640 10 client.?.objecter ms_handle_connect 0x7f2960157b50 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2946ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2946ffd640 10 client.?.objecter ms_handle_connect 0x7f2960157780 2026-02-25T15:01:43.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.816+0000 7f2946ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:43.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4136124555 0 0) 0x7f2960133250 con 0x7f2960164d00 2026-02-25T15:01:43.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 10 monclient(hunting): my global_id is 4566 2026-02-25T15:01:43.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2930003610 con 0x7f2960164d00 2026-02-25T15:01:43.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1484868997 0 0) 0x7f2960133530 con 0x7f2960157780 2026-02-25T15:01:43.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 10 monclient(hunting): my global_id is 4543 2026-02-25T15:01:43.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2960133250 con 0x7f2960157780 2026-02-25T15:01:43.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3797016959 0 0) 0x7f2960133810 con 0x7f2960157b50 2026-02-25T15:01:43.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 10 monclient(hunting): my global_id is 4568 2026-02-25T15:01:43.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.817+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2960133530 con 0x7f2960157b50 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 673424027 0 0) 0x7f2930003610 con 0x7f2960164d00 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2960133810 con 0x7f2960164d00 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f295c0033c0 con 0x7f2960164d00 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2243805111 0 0) 0x7f2960133530 con 0x7f2960157b50 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2930003610 con 0x7f2960157b50 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f29540030b0 con 0x7f2960157b50 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.821 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2259134565 0 0) 0x7f2960133250 con 0x7f2960157780 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2960133530 con 0x7f2960157780 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f2950003310 con 0x7f2960157780 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.820+0000 7f2946ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4252984633 0 0) 0x7f2960133810 con 0x7f2960164d00 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 >> v1:10.20.193.94:6789/0 conn(0x7f2960157b50 legacy=0x7f2960147980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 >> v1:10.20.193.93:6789/0 conn(0x7f2960157780 legacy=0x7f29601329f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2946ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2946ffd640 1 monclient: found mon.a 2026-02-25T15:01:43.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2946ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2946ffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2960133af0 con 0x7f2960164d00 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2946ffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2946ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2946ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2968771640 5 monclient: authenticate success, global_id 4566 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2968771640 10 monclient: _renew_subs 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2968771640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2968771640 1 -- 10.20.193.90:0/3056836534 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2960147460 con 0x7f2960164d00 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2968771640 10 client.4566.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2968771640 10 monclient: _renew_subs 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2968771640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2968771640 1 -- 10.20.193.90:0/3056836534 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2960148270 con 0x7f2960164d00 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.821+0000 7f2968771640 1 librados: init done 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.822+0000 7f2968771640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.822+0000 7f2968771640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.822+0000 7f2968771640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.822+0000 7f2968771640 1 -- 10.20.193.90:0/3056836534 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f296005cff0 con 0x7f2960164d00 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.830+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f295c003a80 con 0x7f2960164d00 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.830+0000 7f2946ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.830+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f295c003e50 con 0x7f2960164d00 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.830+0000 7f2946ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.830+0000 7f2964ce3640 4 set_mon_vals no callback set 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.830+0000 7f2946ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.830+0000 7f2946ffd640 10 monclient: dump: 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.830+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f295c004030 con 0x7f2960164d00 2026-02-25T15:01:43.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.831+0000 7f29664e6640 10 monclient: get_auth_request con 0x7f293007a110 auth_method 0 2026-02-25T15:01:43.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.832+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f295c095d40 con 0x7f2960164d00 2026-02-25T15:01:43.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.832+0000 7f2946ffd640 10 client.4566.objecter ms_dispatch2 0x7f2960000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:43.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.832+0000 7f2946ffd640 3 client.4566.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:43.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.832+0000 7f2946ffd640 3 client.4566.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:43.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.832+0000 7f2946ffd640 20 client.4566.objecter dump_active .. 0 homeless 2026-02-25T15:01:43.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.832+0000 7f2946ffd640 10 client.4566.objecter ms_handle_connect 0x7f293007a110 2026-02-25T15:01:43.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.852+0000 7f08dc31a640 10 monclient: get_monmap_and_config 2026-02-25T15:01:43.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.852+0000 7f08dc31a640 10 monclient: build_initial_monmap 2026-02-25T15:01:43.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.852+0000 7f08dc31a640 10 monclient: monmap: 2026-02-25T15:01:43.853 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:43.853 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.853 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:43.853405+0000 2026-02-25T15:01:43.853 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:43.853405+0000 2026-02-25T15:01:43.853 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:43.853 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.853 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:43.853 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.853+0000 7f08dc31a640 1 Processor -- start 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.853+0000 7f08dc31a640 1 -- start start 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.853+0000 7f08dc31a640 10 monclient: init 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.853+0000 7f08dc31a640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.853+0000 7f08dc31a640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.853+0000 7f08dc31a640 10 monclient(hunting): picked mon.noname-b con 0x7f08d4157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.853+0000 7f08dc31a640 10 monclient(hunting): picked mon.noname-c con 0x7f08d4157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.853+0000 7f08dc31a640 10 monclient(hunting): picked mon.noname-a con 0x7f08d4164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.853+0000 7f08dc31a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d405de70 con 0x7f08d4164d00 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.853+0000 7f08dc31a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d4055b20 con 0x7f08d4157780 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.853+0000 7f08dc31a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d4055e00 con 0x7f08d4157b50 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.853+0000 7f08dc31a640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.853+0000 7f08dc31a640 10 monclient(hunting): authenticate will time out at 2993.453728s 2026-02-25T15:01:43.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.854+0000 7f08da08f640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f08d4157780 0x7f08d4056ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:50990/0 (socket says 10.20.193.90:50990) 2026-02-25T15:01:43.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.854+0000 7f08da08f640 1 -- 10.20.193.90:0/4188781301 learned_addr learned my addr 10.20.193.90:0/4188781301 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:43.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.855+0000 7f08d908d640 1 -- 10.20.193.90:0/4188781301 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3957880241 0 0) 0x7f08d4055b20 con 0x7f08d4157780 2026-02-25T15:01:43.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.855+0000 7f08d908d640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.855+0000 7f08d908d640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.855+0000 7f08d908d640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.855+0000 7f08d908d640 10 monclient(hunting): my global_id is 4546 2026-02-25T15:01:43.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.855+0000 7f08d908d640 1 -- 10.20.193.90:0/4188781301 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08bc003610 con 0x7f08d4157780 2026-02-25T15:01:43.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.855+0000 7f08d908d640 1 -- 10.20.193.90:0/4188781301 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 858209734 0 0) 0x7f08d4055e00 con 0x7f08d4157b50 2026-02-25T15:01:43.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.855+0000 7f08d908d640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.855+0000 7f08d908d640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.855+0000 7f08d908d640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.855+0000 7f08d908d640 10 monclient(hunting): my global_id is 4571 2026-02-25T15:01:43.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.855+0000 7f08d908d640 1 -- 10.20.193.90:0/4188781301 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08d4055b20 con 0x7f08d4157b50 2026-02-25T15:01:43.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08d908d640 1 -- 10.20.193.90:0/4188781301 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1615259802 0 0) 0x7f08bc003610 con 0x7f08d4157780 2026-02-25T15:01:43.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08d908d640 1 -- 10.20.193.90:0/4188781301 >> v1:10.20.193.94:6789/0 conn(0x7f08d4157b50 legacy=0x7f08d41645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08d908d640 1 -- 10.20.193.90:0/4188781301 >> v1:10.20.193.90:6789/0 conn(0x7f08d4164d00 legacy=0x7f08d41c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:43.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08d908d640 10 monclient: _finish_hunting 0 2026-02-25T15:01:43.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08d908d640 1 monclient: found mon.noname-b 2026-02-25T15:01:43.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08d908d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:43.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08d908d640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08d908d640 1 -- 10.20.193.90:0/4188781301 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d40560e0 con 0x7f08d4157780 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08d908d640 10 monclient: _finish_auth 0 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08d908d640 10 monclient: _check_auth_tickets 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08d908d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08d908d640 1 -- 10.20.193.90:0/4188781301 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f08c4002df0 con 0x7f08d4157780 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08dc31a640 5 monclient: authenticate success, global_id 4546 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08dc31a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08d908d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08d908d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08d908d640 10 monclient: dump: 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.856+0000 7f08dc31a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.857+0000 7f08d908d640 1 -- 10.20.193.90:0/4188781301 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f08c4003000 con 0x7f08d4157780 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.857+0000 7f08d908d640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.857+0000 7f08d908d640 1 -- 10.20.193.90:0/4188781301 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f08c4003440 con 0x7f08d4157780 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.857+0000 7f08d908d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.857+0000 7f08d908d640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.857+0000 7f08d908d640 10 monclient: dump: 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.858 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.857+0000 7f08dc31a640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.857+0000 7f08dc31a640 4 set_mon_vals no callback set 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.857+0000 7f08dc31a640 10 monclient: shutdown 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.857+0000 7f08dc31a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.857+0000 7f08dc31a640 1 -- 10.20.193.90:0/4188781301 >> v1:10.20.193.93:6789/0 conn(0x7f08d4157780 legacy=0x7f08d4056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.857+0000 7f08dc31a640 1 -- 10.20.193.90:0/4188781301 shutdown_connections 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.857+0000 7f08dc31a640 1 -- 10.20.193.90:0/4188781301 >> 10.20.193.90:0/4188781301 conn(0x7f08d40b2ef0 msgr2=0x7f08d40b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.857+0000 7f08dc31a640 1 -- 10.20.193.90:0/4188781301 shutdown_connections 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.857+0000 7f08dc31a640 1 -- 10.20.193.90:0/4188781301 wait complete. 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.858+0000 7f08dc31a640 10 monclient: build_initial_monmap 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.858+0000 7f08dc31a640 10 monclient: monmap: 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:43.859 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.858+0000 7f08dc31a640 1 librados: starting msgr at 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.858+0000 7f08dc31a640 1 librados: starting objecter 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.858+0000 7f08dc31a640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.858+0000 7f08dc31a640 1 Processor -- start 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.858+0000 7f08dc31a640 1 -- start start 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.858+0000 7f08dc31a640 1 librados: setting wanted keys 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.858+0000 7f08dc31a640 1 librados: calling monclient init 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.858+0000 7f08dc31a640 10 monclient: init 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.859+0000 7f08dc31a640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.859+0000 7f08dc31a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.859+0000 7f08dc31a640 10 monclient(hunting): picked mon.noname-c con 0x7f08d4157780 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.859+0000 7f08dc31a640 10 monclient(hunting): picked mon.noname-b con 0x7f08d4157b50 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.859+0000 7f08dc31a640 10 monclient(hunting): picked mon.noname-a con 0x7f08d4164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.859+0000 7f08dc31a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d41fd1a0 con 0x7f08d4164d00 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.859+0000 7f08dc31a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d41fd460 con 0x7f08d4157b50 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.859+0000 7f08dc31a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d41fd740 con 0x7f08d4157780 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.859+0000 7f08dc31a640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.859+0000 7f08dc31a640 10 monclient(hunting): authenticate will time out at 2993.459093s 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.859+0000 7f08d988e640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f08d4157b50 0x7f08d40afb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:51000/0 (socket says 10.20.193.90:51000) 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.859+0000 7f08d988e640 1 -- 10.20.193.90:0/600907781 learned_addr learned my addr 10.20.193.90:0/600907781 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.859+0000 7f08baffd640 10 client.?.objecter ms_handle_connect 0x7f08d4157b50 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.859+0000 7f08baffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.860+0000 7f08baffd640 10 client.?.objecter ms_handle_connect 0x7f08d4157780 2026-02-25T15:01:43.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.860+0000 7f08baffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:43.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.860+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 87622078 0 0) 0x7f08d41fd460 con 0x7f08d4157b50 2026-02-25T15:01:43.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.860+0000 7f08baffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.860+0000 7f08baffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.860+0000 7f08baffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.860+0000 7f08baffd640 10 monclient(hunting): my global_id is 4549 2026-02-25T15:01:43.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.860+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08a4003610 con 0x7f08d4157b50 2026-02-25T15:01:43.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.861+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2397733983 0 0) 0x7f08d41fd740 con 0x7f08d4157780 2026-02-25T15:01:43.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.861+0000 7f08baffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.861+0000 7f08baffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.862 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.861+0000 7f08baffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.862 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.861+0000 7f08baffd640 10 monclient(hunting): my global_id is 4574 2026-02-25T15:01:43.862 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.861+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08d41fd460 con 0x7f08d4157780 2026-02-25T15:01:43.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.862+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4166850821 0 0) 0x7f08a4003610 con 0x7f08d4157b50 2026-02-25T15:01:43.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.862+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f08d41fd740 con 0x7f08d4157b50 2026-02-25T15:01:43.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.862+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f08c8003170 con 0x7f08d4157b50 2026-02-25T15:01:43.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.862+0000 7f08baffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:43.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.862+0000 7f08baffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:43.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.862+0000 7f08baffd640 10 monclient(hunting): dump: 2026-02-25T15:01:43.863 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.863 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.863 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.863 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.863 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.863 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.863 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.863 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.862+0000 7f08baffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.863+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1195547197 0 0) 0x7f08d41fd460 con 0x7f08d4157780 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.863+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f08a4003610 con 0x7f08d4157780 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.863+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f08c4003350 con 0x7f08d4157780 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.863+0000 7f08baffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.863+0000 7f08baffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.863+0000 7f08baffd640 10 monclient(hunting): dump: 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.864 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.865 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4050968063 0 0) 0x7f08d41fd740 con 0x7f08d4157b50 2026-02-25T15:01:43.865 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 >> v1:10.20.193.94:6789/0 conn(0x7f08d4157780 legacy=0x7f08d41f91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.865 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 >> v1:10.20.193.90:6789/0 conn(0x7f08d4164d00 legacy=0x7f08d40a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.865 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08baffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:43.865 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08baffd640 1 monclient: found mon.b 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08baffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08baffd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d41fda20 con 0x7f08d4157b50 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08baffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08baffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08baffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08dc31a640 5 monclient: authenticate success, global_id 4549 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08dc31a640 10 monclient: _renew_subs 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08dc31a640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08dc31a640 1 -- 10.20.193.90:0/600907781 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f08d41fdd00 con 0x7f08d4157b50 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08dc31a640 10 client.4549.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08dc31a640 10 monclient: _renew_subs 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08dc31a640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08dc31a640 1 -- 10.20.193.90:0/600907781 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f08d41f8cd0 con 0x7f08d4157b50 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.864+0000 7f08dc31a640 1 librados: init done 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.865+0000 7f08dc31a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.865+0000 7f08dc31a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.865+0000 7f08dc31a640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:43.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.865+0000 7f08dc31a640 1 -- 10.20.193.90:0/600907781 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d40531e0 con 0x7f08d4157b50 2026-02-25T15:01:43.875 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-25T15:01:43.917 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.916+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f295c05f6a0 con 0x7f2960164d00 2026-02-25T15:01:43.917 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.916+0000 7f2946ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.917 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.916+0000 7f2946ffd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:43.940 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.939+0000 7f145d31a640 10 monclient: get_monmap_and_config 2026-02-25T15:01:43.940 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.939+0000 7f145d31a640 10 monclient: build_initial_monmap 2026-02-25T15:01:43.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.939+0000 7f145d31a640 10 monclient: monmap: 2026-02-25T15:01:43.941 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:43.941 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.941 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:43.940628+0000 2026-02-25T15:01:43.941 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:43.940628+0000 2026-02-25T15:01:43.941 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:43.941 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.941 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:43.941 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:43.941 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:43.941 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.940+0000 7f145d31a640 1 Processor -- start 2026-02-25T15:01:43.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.940+0000 7f145d31a640 1 -- start start 2026-02-25T15:01:43.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.940+0000 7f145d31a640 10 monclient: init 2026-02-25T15:01:43.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.940+0000 7f145d31a640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:43.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.941+0000 7f145d31a640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:43.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.941+0000 7f145d31a640 10 monclient(hunting): picked mon.noname-a con 0x7f145815b0a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:43.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.941+0000 7f145d31a640 10 monclient(hunting): picked mon.noname-b con 0x7f14580b3190 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:43.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.941+0000 7f145d31a640 10 monclient(hunting): picked mon.noname-c con 0x7f14580b5da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:43.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.941+0000 7f145d31a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14580b3610 con 0x7f145815b0a0 2026-02-25T15:01:43.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.941+0000 7f145d31a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14580b38f0 con 0x7f14580b3190 2026-02-25T15:01:43.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.941+0000 7f145d31a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14580b3bd0 con 0x7f14580b5da0 2026-02-25T15:01:43.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.941+0000 7f145d31a640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:43.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.941+0000 7f145d31a640 10 monclient(hunting): authenticate will time out at 2993.541079s 2026-02-25T15:01:43.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.941+0000 7f1456ffd640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f145815b0a0 0x7f1458162300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57202/0 (socket says 10.20.193.90:57202) 2026-02-25T15:01:43.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.941+0000 7f1456ffd640 1 -- 10.20.193.90:0/2040812844 learned_addr learned my addr 10.20.193.90:0/2040812844 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:43.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.942+0000 7f1455ffb640 1 -- 10.20.193.90:0/2040812844 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 165059117 0 0) 0x7f14580b3bd0 con 0x7f14580b5da0 2026-02-25T15:01:43.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.942+0000 7f1455ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.942+0000 7f1455ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.942+0000 7f1455ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.942+0000 7f1455ffb640 10 monclient(hunting): my global_id is 4577 2026-02-25T15:01:43.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.942+0000 7f1455ffb640 1 -- 10.20.193.90:0/2040812844 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1444003610 con 0x7f14580b5da0 2026-02-25T15:01:43.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.942+0000 7f1455ffb640 1 -- 10.20.193.90:0/2040812844 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3279053675 0 0) 0x7f14580b3610 con 0x7f145815b0a0 2026-02-25T15:01:43.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.942+0000 7f1455ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.942+0000 7f1455ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.942+0000 7f1455ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.942+0000 7f1455ffb640 10 monclient(hunting): my global_id is 4569 2026-02-25T15:01:43.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.942+0000 7f1455ffb640 1 -- 10.20.193.90:0/2040812844 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14580b3bd0 con 0x7f145815b0a0 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f1455ffb640 1 -- 10.20.193.90:0/2040812844 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1802285953 0 0) 0x7f1444003610 con 0x7f14580b5da0 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f1455ffb640 1 -- 10.20.193.90:0/2040812844 >> v1:10.20.193.93:6789/0 conn(0x7f14580b3190 legacy=0x7f14580b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f1455ffb640 1 -- 10.20.193.90:0/2040812844 >> v1:10.20.193.90:6789/0 conn(0x7f145815b0a0 legacy=0x7f1458162300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f1455ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f1455ffb640 1 monclient: found mon.noname-c 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f1455ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f1455ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f1455ffb640 1 -- 10.20.193.90:0/2040812844 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14580b3eb0 con 0x7f14580b5da0 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f1455ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f1455ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f1455ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f145d31a640 5 monclient: authenticate success, global_id 4577 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f145d31a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f1455ffb640 1 -- 10.20.193.90:0/2040812844 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1448003170 con 0x7f14580b5da0 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f1455ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f1455ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f1455ffb640 10 monclient: dump: 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.944+0000 7f145d31a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f1455ffb640 1 -- 10.20.193.90:0/2040812844 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f14480033c0 con 0x7f14580b5da0 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f1455ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f1455ffb640 1 -- 10.20.193.90:0/2040812844 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1448003800 con 0x7f14580b5da0 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f145d31a640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f145d31a640 4 set_mon_vals no callback set 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f1455ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f1455ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f1455ffb640 10 monclient: dump: 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f145d31a640 10 monclient: shutdown 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f145d31a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f145d31a640 1 -- 10.20.193.90:0/2040812844 >> v1:10.20.193.94:6789/0 conn(0x7f14580b5da0 legacy=0x7f14580b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f145d31a640 1 -- 10.20.193.90:0/2040812844 shutdown_connections 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f145d31a640 1 -- 10.20.193.90:0/2040812844 >> 10.20.193.90:0/2040812844 conn(0x7f14580b5290 msgr2=0x7f14580b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f145d31a640 1 -- 10.20.193.90:0/2040812844 shutdown_connections 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f145d31a640 1 -- 10.20.193.90:0/2040812844 wait complete. 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f145d31a640 10 monclient: build_initial_monmap 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f145d31a640 10 monclient: monmap: 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:43.946 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f145d31a640 1 librados: starting msgr at 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.945+0000 7f145d31a640 1 librados: starting objecter 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 1 Processor -- start 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 1 -- start start 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 1 librados: setting wanted keys 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 1 librados: calling monclient init 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 10 monclient: init 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 10 monclient(hunting): picked mon.noname-c con 0x7f14580b5da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 10 monclient(hunting): picked mon.noname-b con 0x7f145815b0a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 10 monclient(hunting): picked mon.noname-a con 0x7f14581fb2d0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14581ff750 con 0x7f14581fb2d0 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1458135fa0 con 0x7f145815b0a0 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14581ff190 con 0x7f14580b5da0 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:43.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f145d31a640 10 monclient(hunting): authenticate will time out at 2993.546531s 2026-02-25T15:01:43.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f14577fe640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f14581fb2d0 0x7f1458135890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57214/0 (socket says 10.20.193.90:57214) 2026-02-25T15:01:43.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f14577fe640 1 -- 10.20.193.90:0/2140504661 learned_addr learned my addr 10.20.193.90:0/2140504661 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:43.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f14377fe640 10 client.?.objecter ms_handle_connect 0x7f14581fb2d0 2026-02-25T15:01:43.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.946+0000 7f14377fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:43.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.947+0000 7f14377fe640 10 client.?.objecter ms_handle_connect 0x7f14580b5da0 2026-02-25T15:01:43.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.947+0000 7f14377fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:43.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.947+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 802711521 0 0) 0x7f14581ff750 con 0x7f14581fb2d0 2026-02-25T15:01:43.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.947+0000 7f14377fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.947+0000 7f14377fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.947+0000 7f14377fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.947+0000 7f14377fe640 10 monclient(hunting): my global_id is 4572 2026-02-25T15:01:43.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.947+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1424003610 con 0x7f14581fb2d0 2026-02-25T15:01:43.949 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.948+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 548666499 0 0) 0x7f14581ff190 con 0x7f14580b5da0 2026-02-25T15:01:43.949 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.948+0000 7f14377fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:43.949 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.948+0000 7f14377fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:43.949 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.948+0000 7f14377fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:43.949 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.948+0000 7f14377fe640 10 monclient(hunting): my global_id is 4580 2026-02-25T15:01:43.949 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.948+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14581ff750 con 0x7f14580b5da0 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.950+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1730259868 0 0) 0x7f1424003610 con 0x7f14581fb2d0 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.950+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f14581ff190 con 0x7f14581fb2d0 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.950+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1448003710 con 0x7f14581fb2d0 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.950+0000 7f14377fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.950+0000 7f14377fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.950+0000 7f14377fe640 10 monclient(hunting): dump: 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.950+0000 7f14377fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.950+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1708145387 0 0) 0x7f14581ff750 con 0x7f14580b5da0 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.950+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1424003610 con 0x7f14580b5da0 2026-02-25T15:01:43.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.950+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f144c002fb0 con 0x7f14580b5da0 2026-02-25T15:01:43.952 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.950+0000 7f14377fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:43.952 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.950+0000 7f14377fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:43.952 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.950+0000 7f14377fe640 10 monclient(hunting): dump: 2026-02-25T15:01:43.952 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.952 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.952 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.952 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.952 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.952 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.952 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.952 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.952 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.952 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3866989190 0 0) 0x7f14581ff190 con 0x7f14581fb2d0 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 >> v1:10.20.193.94:6789/0 conn(0x7f14580b5da0 legacy=0x7f1458160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 >> v1:10.20.193.93:6789/0 conn(0x7f145815b0a0 legacy=0x7f14581fe850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f14377fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f14377fe640 1 monclient: found mon.a 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f14377fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f14377fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14581ff470 con 0x7f14581fb2d0 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f14377fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f14377fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f14377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f145d31a640 5 monclient: authenticate success, global_id 4572 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f145d31a640 10 monclient: _renew_subs 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f145d31a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f145d31a640 1 -- 10.20.193.90:0/2140504661 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f14580b0740 con 0x7f14581fb2d0 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f145d31a640 10 client.4572.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f145d31a640 10 monclient: _renew_subs 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f145d31a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f145d31a640 1 -- 10.20.193.90:0/2140504661 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f14580b1220 con 0x7f14581fb2d0 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f145d31a640 1 librados: init done 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f145d31a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f145d31a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f145d31a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:43.953 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.952+0000 7f145d31a640 1 -- 10.20.193.90:0/2140504661 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1458055260 con 0x7f14581fb2d0 2026-02-25T15:01:43.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.956+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f08c8003520 con 0x7f08d4157b50 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.956+0000 7f08baffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.956+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f08c8003b50 con 0x7f08d4157b50 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.956+0000 7f08baffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.956+0000 7f08baffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.956+0000 7f08baffd640 10 monclient: dump: 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.956+0000 7f08d888c640 4 set_mon_vals no callback set 2026-02-25T15:01:43.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.956+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f08c8003d30 con 0x7f08d4157b50 2026-02-25T15:01:43.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.957+0000 7f08da08f640 10 monclient: get_auth_request con 0x7f08a4079e60 auth_method 0 2026-02-25T15:01:43.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.957+0000 7f08baffd640 10 client.4549.objecter ms_handle_connect 0x7f08a4079e60 2026-02-25T15:01:43.959 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.958+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f08c8095d90 con 0x7f08d4157b50 2026-02-25T15:01:43.959 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.958+0000 7f08baffd640 10 client.4549.objecter ms_dispatch2 0x7f08d4000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:43.959 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.958+0000 7f08baffd640 3 client.4549.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:43.959 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.958+0000 7f08baffd640 3 client.4549.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:43.959 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.958+0000 7f08baffd640 20 client.4549.objecter dump_active .. 0 homeless 2026-02-25T15:01:43.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.959+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f08d40531e0 con 0x7f08d4157b50 2026-02-25T15:01:43.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.959+0000 7f08baffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:43.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:43.959+0000 7f08baffd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:43.976 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-25T15:01:43.976 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2026-02-25T15:01:43.982 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2026-02-25T15:01:44.025 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.024+0000 7f2968771640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-25T15:01:44.025 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.024+0000 7f2968771640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-25T15:01:44.025 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.024+0000 7f2968771640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.026 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.024+0000 7f2968771640 1 -- 10.20.193.90:0/3056836534 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f296005cd10 con 0x7f2960164d00 2026-02-25T15:01:44.044 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.043+0000 7f193209d640 10 monclient: get_monmap_and_config 2026-02-25T15:01:44.044 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.043+0000 7f193209d640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.044 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.043+0000 7f193209d640 10 monclient: monmap: 2026-02-25T15:01:44.044 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.044 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.045 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:44.043848+0000 2026-02-25T15:01:44.045 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:44.043848+0000 2026-02-25T15:01:44.045 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.045 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.045 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.045 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.045 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.045 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.046 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.043+0000 7f193209d640 1 Processor -- start 2026-02-25T15:01:44.046 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.043+0000 7f193209d640 1 -- start start 2026-02-25T15:01:44.046 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.043+0000 7f193209d640 10 monclient: init 2026-02-25T15:01:44.046 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.043+0000 7f10d6488640 10 monclient: get_monmap_and_config 2026-02-25T15:01:44.047 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.043+0000 7f10d6488640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.047 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.043+0000 7f10d6488640 10 monclient: monmap: 2026-02-25T15:01:44.047 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.047 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.047 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:44.044395+0000 2026-02-25T15:01:44.047 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:44.044395+0000 2026-02-25T15:01:44.047 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.047 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.047 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.047 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.047 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.047 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.048 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f193209d640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.049 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f193209d640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:44.049 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f193209d640 10 monclient(hunting): picked mon.noname-b con 0x7f192c157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.049 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f193209d640 10 monclient(hunting): picked mon.noname-a con 0x7f192c157b50 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.049 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f193209d640 10 monclient(hunting): picked mon.noname-c con 0x7f192c164d00 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.049 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f193209d640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f192c05de70 con 0x7f192c157b50 2026-02-25T15:01:44.049 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f193209d640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f192c055b20 con 0x7f192c157780 2026-02-25T15:01:44.049 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f193209d640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f192c055e00 con 0x7f192c164d00 2026-02-25T15:01:44.049 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f193209d640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.049 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f193209d640 10 monclient(hunting): authenticate will time out at 2993.644194s 2026-02-25T15:01:44.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f10d6488640 1 Processor -- start 2026-02-25T15:01:44.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f10d6488640 1 -- start start 2026-02-25T15:01:44.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f10d6488640 10 monclient: init 2026-02-25T15:01:44.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f192bfff640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f192c164d00 0x7f192c1c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45650/0 (socket says 10.20.193.90:45650) 2026-02-25T15:01:44.050 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f192bfff640 1 -- 10.20.193.90:0/1344781504 learned_addr learned my addr 10.20.193.90:0/1344781504 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.051 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f10d6488640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.051 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f10d6488640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:44.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f10d6488640 10 monclient(hunting): picked mon.noname-c con 0x7f10d00b9f80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f10d6488640 10 monclient(hunting): picked mon.noname-b con 0x7f10d00ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f10d6488640 10 monclient(hunting): picked mon.noname-a con 0x7f10d00585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f10d6488640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d0163930 con 0x7f10d00585e0 2026-02-25T15:01:44.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f10d6488640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d0163bb0 con 0x7f10d00ba350 2026-02-25T15:01:44.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f10d6488640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d0163e90 con 0x7f10d00b9f80 2026-02-25T15:01:44.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f10d6488640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.044+0000 7f10d6488640 10 monclient(hunting): authenticate will time out at 2993.644804s 2026-02-25T15:01:44.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.045+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1448003360 con 0x7f14581fb2d0 2026-02-25T15:01:44.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.045+0000 7f14377fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.045+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1448003f70 con 0x7f14581fb2d0 2026-02-25T15:01:44.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.045+0000 7f14377fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.045+0000 7f14377fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:44.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.045+0000 7f14377fe640 10 monclient: dump: 2026-02-25T15:01:44.053 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.053 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.053 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.053 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.054 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.054 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.054 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.054 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.054 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.054 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.054 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.045+0000 7f14557fa640 4 set_mon_vals no callback set 2026-02-25T15:01:44.055 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.045+0000 7f10cf7fe640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f10d00ba350 0x7f10d0057ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:51040/0 (socket says 10.20.193.90:51040) 2026-02-25T15:01:44.055 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.045+0000 7f10cf7fe640 1 -- 10.20.193.90:0/3749096473 learned_addr learned my addr 10.20.193.90:0/3749096473 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.055 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.045+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f1448004110 con 0x7f14581fb2d0 2026-02-25T15:01:44.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.046+0000 7f192a7fc640 1 -- 10.20.193.90:0/1344781504 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1430157662 0 0) 0x7f192c055e00 con 0x7f192c164d00 2026-02-25T15:01:44.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.046+0000 7f192a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.046+0000 7f192a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.046+0000 7f192a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.046+0000 7f192a7fc640 10 monclient(hunting): my global_id is 4583 2026-02-25T15:01:44.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.046+0000 7f192a7fc640 1 -- 10.20.193.90:0/1344781504 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1910003610 con 0x7f192c164d00 2026-02-25T15:01:44.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.046+0000 7f1456ffd640 10 monclient: get_auth_request con 0x7f1424079e90 auth_method 0 2026-02-25T15:01:44.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.046+0000 7f14377fe640 10 client.4572.objecter ms_handle_connect 0x7f1424079e90 2026-02-25T15:01:44.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.046+0000 7f192a7fc640 1 -- 10.20.193.90:0/1344781504 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 71519453 0 0) 0x7f192c055b20 con 0x7f192c157780 2026-02-25T15:01:44.057 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.046+0000 7f192a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.046+0000 7f192a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.046+0000 7f192a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.046+0000 7f192a7fc640 10 monclient(hunting): my global_id is 4552 2026-02-25T15:01:44.058 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.046+0000 7f192a7fc640 1 -- 10.20.193.90:0/1344781504 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f192c055e00 con 0x7f192c157780 2026-02-25T15:01:44.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.047+0000 7f10ceffd640 1 -- 10.20.193.90:0/3749096473 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3438941001 0 0) 0x7f10d0163bb0 con 0x7f10d00ba350 2026-02-25T15:01:44.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.047+0000 7f10ceffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.047+0000 7f10ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.047+0000 7f10ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.047+0000 7f10ceffd640 10 monclient(hunting): my global_id is 4555 2026-02-25T15:01:44.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.047+0000 7f10ceffd640 1 -- 10.20.193.90:0/3749096473 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10b4003610 con 0x7f10d00ba350 2026-02-25T15:01:44.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.048+0000 7f10ceffd640 1 -- 10.20.193.90:0/3749096473 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3550026285 0 0) 0x7f10d0163e90 con 0x7f10d00b9f80 2026-02-25T15:01:44.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.048+0000 7f10ceffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.048+0000 7f10ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.048+0000 7f10ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.059 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.048+0000 7f10ceffd640 10 monclient(hunting): my global_id is 4586 2026-02-25T15:01:44.060 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.048+0000 7f10ceffd640 1 -- 10.20.193.90:0/3749096473 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10d0163bb0 con 0x7f10d00b9f80 2026-02-25T15:01:44.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f192a7fc640 1 -- 10.20.193.90:0/1344781504 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1751665803 0 0) 0x7f1910003610 con 0x7f192c164d00 2026-02-25T15:01:44.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f192a7fc640 1 -- 10.20.193.90:0/1344781504 >> v1:10.20.193.93:6789/0 conn(0x7f192c157780 legacy=0x7f192c056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f192a7fc640 1 -- 10.20.193.90:0/1344781504 >> v1:10.20.193.90:6789/0 conn(0x7f192c157b50 legacy=0x7f192c1645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:44.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f192a7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f192a7fc640 1 monclient: found mon.noname-c 2026-02-25T15:01:44.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f192a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f192a7fc640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f192a7fc640 1 -- 10.20.193.90:0/1344781504 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192c0560e0 con 0x7f192c164d00 2026-02-25T15:01:44.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f192a7fc640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f192a7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.062 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f192a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.062 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f192a7fc640 1 -- 10.20.193.90:0/1344781504 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1920003170 con 0x7f192c164d00 2026-02-25T15:01:44.062 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f192a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.062 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f192a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:44.062 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f192a7fc640 10 monclient: dump: 2026-02-25T15:01:44.062 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.062 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.062 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.062 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.062 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.062 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.062 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.062 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.063 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.063 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.063 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f193209d640 5 monclient: authenticate success, global_id 4583 2026-02-25T15:01:44.063 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f193209d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.063 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f1448095f40 con 0x7f14581fb2d0 2026-02-25T15:01:44.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f14377fe640 10 client.4572.objecter ms_dispatch2 0x7f1458000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:44.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f14377fe640 3 client.4572.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:44.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f14377fe640 3 client.4572.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:44.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f14377fe640 20 client.4572.objecter dump_active .. 0 homeless 2026-02-25T15:01:44.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f1458055260 con 0x7f14581fb2d0 2026-02-25T15:01:44.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f14377fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.049+0000 7f14377fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:44.065 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.050+0000 7f10ceffd640 1 -- 10.20.193.90:0/3749096473 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1295679643 0 0) 0x7f10d0163bb0 con 0x7f10d00b9f80 2026-02-25T15:01:44.065 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.050+0000 7f10ceffd640 1 -- 10.20.193.90:0/3749096473 >> v1:10.20.193.93:6789/0 conn(0x7f10d00ba350 legacy=0x7f10d0057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.065 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.050+0000 7f10ceffd640 1 -- 10.20.193.90:0/3749096473 >> v1:10.20.193.90:6789/0 conn(0x7f10d00585e0 legacy=0x7f10d0166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:44.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.050+0000 7f10ceffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.050+0000 7f10ceffd640 1 monclient: found mon.noname-c 2026-02-25T15:01:44.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.050+0000 7f10ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.050+0000 7f10ceffd640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.050+0000 7f10ceffd640 1 -- 10.20.193.90:0/3749096473 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d00573f0 con 0x7f10d00b9f80 2026-02-25T15:01:44.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.050+0000 7f10ceffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.050+0000 7f10ceffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.050+0000 7f10ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.050+0000 7f10d6488640 5 monclient: authenticate success, global_id 4586 2026-02-25T15:01:44.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.050+0000 7f10d6488640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f10ceffd640 1 -- 10.20.193.90:0/3749096473 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f10bc002df0 con 0x7f10d00b9f80 2026-02-25T15:01:44.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f10ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f10ceffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:44.067 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f10ceffd640 10 monclient: dump: 2026-02-25T15:01:44.067 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.067 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.067 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.067 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.067 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.067 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.067 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.067 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.067 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.067 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.068 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f10d6488640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.069 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f192a7fc640 1 -- 10.20.193.90:0/1344781504 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1920003380 con 0x7f192c164d00 2026-02-25T15:01:44.069 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f192a7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.069 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f192a7fc640 1 -- 10.20.193.90:0/1344781504 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f19200037c0 con 0x7f192c164d00 2026-02-25T15:01:44.069 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f192a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f192a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:44.070 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f192a7fc640 10 monclient: dump: 2026-02-25T15:01:44.070 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.070 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.070 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.070 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.070 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.070 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.070 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.070 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.070 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.070 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f193209d640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:44.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f193209d640 4 set_mon_vals no callback set 2026-02-25T15:01:44.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f193209d640 10 monclient: shutdown 2026-02-25T15:01:44.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f193209d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.051+0000 7f193209d640 1 -- 10.20.193.90:0/1344781504 >> v1:10.20.193.94:6789/0 conn(0x7f192c164d00 legacy=0x7f192c1c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f193209d640 1 -- 10.20.193.90:0/1344781504 shutdown_connections 2026-02-25T15:01:44.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f193209d640 1 -- 10.20.193.90:0/1344781504 >> 10.20.193.90:0/1344781504 conn(0x7f192c0b2ef0 msgr2=0x7f192c0b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f193209d640 1 -- 10.20.193.90:0/1344781504 shutdown_connections 2026-02-25T15:01:44.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f193209d640 1 -- 10.20.193.90:0/1344781504 wait complete. 2026-02-25T15:01:44.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f193209d640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.071 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f193209d640 10 monclient: monmap: 2026-02-25T15:01:44.071 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.071 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:44.072 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:44.072 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:44.072 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.072 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:44.072 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.072 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.072 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.072 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.072 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f193209d640 1 librados: starting msgr at 2026-02-25T15:01:44.072 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f193209d640 1 librados: starting objecter 2026-02-25T15:01:44.072 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f193209d640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:44.072 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f193209d640 1 Processor -- start 2026-02-25T15:01:44.072 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f193209d640 1 -- start start 2026-02-25T15:01:44.072 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f193209d640 1 librados: setting wanted keys 2026-02-25T15:01:44.073 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f193209d640 1 librados: calling monclient init 2026-02-25T15:01:44.073 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f193209d640 10 monclient: init 2026-02-25T15:01:44.073 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f2946ffd640 1 -- 10.20.193.90:0/3056836534 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (unknown 832126871 0 778662779) 0x7f295c003e50 con 0x7f2960164d00 2026-02-25T15:01:44.073 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f2946ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-25T15:01:44.073 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f2946ffd640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f10ceffd640 1 -- 10.20.193.90:0/3749096473 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f10bc003080 con 0x7f10d00b9f80 2026-02-25T15:01:44.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f10ceffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f10ceffd640 1 -- 10.20.193.90:0/3749096473 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f10bc0034c0 con 0x7f10d00b9f80 2026-02-25T15:01:44.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f10ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f10ceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:44.074 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f10ceffd640 10 monclient: dump: 2026-02-25T15:01:44.074 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.074 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.074 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.074 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.074 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.074 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.075 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.075 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.075 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.075 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.075 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f10d6488640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:44.075 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.052+0000 7f10d6488640 4 set_mon_vals no callback set 2026-02-25T15:01:44.075 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:44.076 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f193209d640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.076 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f193209d640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:44.076 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f193209d640 10 monclient(hunting): picked mon.noname-b con 0x7f192c157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.076 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f193209d640 10 monclient(hunting): picked mon.noname-a con 0x7f192c157b50 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.076 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f193209d640 10 monclient(hunting): picked mon.noname-c con 0x7f192c164d00 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.076 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f193209d640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f192c0ac290 con 0x7f192c157b50 2026-02-25T15:01:44.076 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f193209d640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f192c0b04d0 con 0x7f192c157780 2026-02-25T15:01:44.076 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f193209d640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f192c12b190 con 0x7f192c164d00 2026-02-25T15:01:44.077 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f193209d640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.077 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f193209d640 10 monclient(hunting): authenticate will time out at 2993.653110s 2026-02-25T15:01:44.077 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f10d6488640 10 monclient: shutdown 2026-02-25T15:01:44.077 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f10d6488640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.077 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f10d6488640 1 -- 10.20.193.90:0/3749096473 >> v1:10.20.193.94:6789/0 conn(0x7f10d00b9f80 legacy=0x7f10d0163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.077 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f10d6488640 1 -- 10.20.193.90:0/3749096473 shutdown_connections 2026-02-25T15:01:44.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f10d6488640 1 -- 10.20.193.90:0/3749096473 >> 10.20.193.90:0/3749096473 conn(0x7f10d00b7990 msgr2=0x7f10d00b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f10d6488640 1 -- 10.20.193.90:0/3749096473 shutdown_connections 2026-02-25T15:01:44.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f10d6488640 1 -- 10.20.193.90:0/3749096473 wait complete. 2026-02-25T15:01:44.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f192b7fe640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f192c157780 0x7f192c0afdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:51046/0 (socket says 10.20.193.90:51046) 2026-02-25T15:01:44.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f192b7fe640 1 -- 10.20.193.90:0/2792121353 learned_addr learned my addr 10.20.193.90:0/2792121353 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f1907fff640 10 client.?.objecter ms_handle_connect 0x7f192c157780 2026-02-25T15:01:44.078 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f1907fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.079 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f10d6488640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.079 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.053+0000 7f10d6488640 10 monclient: monmap: 2026-02-25T15:01:44.079 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.079 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:44.079 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:44.079 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:44.079 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.079 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:44.080 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.080 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.080 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.080 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 1 librados: starting msgr at 2026-02-25T15:01:44.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 1 librados: starting objecter 2026-02-25T15:01:44.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:44.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 1 Processor -- start 2026-02-25T15:01:44.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 1 -- start start 2026-02-25T15:01:44.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 1 librados: setting wanted keys 2026-02-25T15:01:44.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 1 librados: calling monclient init 2026-02-25T15:01:44.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 10 monclient: init 2026-02-25T15:01:44.081 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3205493113 0 0) 0x7f192c0b04d0 con 0x7f192c157780 2026-02-25T15:01:44.081 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f1907fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.081 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f1907fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.081 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f1907fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.081 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f1907fff640 10 monclient(hunting): my global_id is 4558 2026-02-25T15:01:44.081 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18fc003610 con 0x7f192c157780 2026-02-25T15:01:44.081 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.081 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:44.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 10 monclient(hunting): picked mon.noname-b con 0x7f10d00585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 10 monclient(hunting): picked mon.noname-c con 0x7f10d00b9f80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 10 monclient(hunting): picked mon.noname-a con 0x7f10d00ba350 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d015eb80 con 0x7f10d00ba350 2026-02-25T15:01:44.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d015ee60 con 0x7f10d00585e0 2026-02-25T15:01:44.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d015f140 con 0x7f10d00b9f80 2026-02-25T15:01:44.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f10d6488640 10 monclient(hunting): authenticate will time out at 2993.654743s 2026-02-25T15:01:44.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f1907fff640 10 client.?.objecter ms_handle_connect 0x7f192c164d00 2026-02-25T15:01:44.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.054+0000 7f1907fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.082 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.055+0000 7f2968771640 10 librados: watch_flush enter 2026-02-25T15:01:44.083 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.055+0000 7f2968771640 10 librados: watch_flush exit 2026-02-25T15:01:44.083 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.055+0000 7f2968771640 20 client.4566.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.083 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.055+0000 7f2968771640 10 client.4566.objecter successfully canceled tick 2026-02-25T15:01:44.083 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.055+0000 7f1907fff640 10 client.?.objecter ms_handle_connect 0x7f192c157b50 2026-02-25T15:01:44.083 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.055+0000 7f1907fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.084 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.055+0000 7f2968771640 1 -- 10.20.193.90:0/3056836534 >> v1:10.20.193.189:6800/3647375982 conn(0x7f293007a110 legacy=0x7f293009a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.084 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.055+0000 7f2968771640 10 monclient: shutdown 2026-02-25T15:01:44.084 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.055+0000 7f2968771640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.084 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.055+0000 7f2968771640 1 -- 10.20.193.90:0/3056836534 >> v1:10.20.193.90:6789/0 conn(0x7f2960164d00 legacy=0x7f2960139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.084 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.055+0000 7f2968771640 1 -- 10.20.193.90:0/3056836534 shutdown_connections 2026-02-25T15:01:44.084 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.055+0000 7f2968771640 1 -- 10.20.193.90:0/3056836534 >> 10.20.193.90:0/3056836534 conn(0x7f29600b2ef0 msgr2=0x7f296005b740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.084 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.055+0000 7f2968771640 1 -- 10.20.193.90:0/3056836534 shutdown_connections 2026-02-25T15:01:44.084 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.055+0000 7f2968771640 1 -- 10.20.193.90:0/3056836534 wait complete. 2026-02-25T15:01:44.084 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.055+0000 7f2968771640 1 librados: shutdown 2026-02-25T15:01:44.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.056+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2942406529 0 0) 0x7f192c12b190 con 0x7f192c164d00 2026-02-25T15:01:44.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.056+0000 7f1907fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.056+0000 7f1907fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.056+0000 7f1907fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.056+0000 7f1907fff640 10 monclient(hunting): my global_id is 4589 2026-02-25T15:01:44.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.056+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f192c0b04d0 con 0x7f192c164d00 2026-02-25T15:01:44.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.056+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3669884236 0 0) 0x7f18fc003610 con 0x7f192c157780 2026-02-25T15:01:44.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.056+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f192c12b190 con 0x7f192c157780 2026-02-25T15:01:44.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.056+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1918003170 con 0x7f192c157780 2026-02-25T15:01:44.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.056+0000 7f1907fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.056+0000 7f10cf7fe640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f10d00b9f80 0x7f10d01fc0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45676/0 (socket says 10.20.193.90:45676) 2026-02-25T15:01:44.085 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.056+0000 7f10cf7fe640 1 -- 10.20.193.90:0/235727452 learned_addr learned my addr 10.20.193.90:0/235727452 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.056+0000 7f1907fff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:44.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.056+0000 7f1907fff640 10 monclient(hunting): dump: 2026-02-25T15:01:44.086 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.086 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.086 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.086 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.086 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.086 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.086 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.086 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.086 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.086 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.086 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.056+0000 7f1907fff640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:44.087 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.057+0000 7f10ccff9640 10 client.?.objecter ms_handle_connect 0x7f10d00b9f80 2026-02-25T15:01:44.087 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.057+0000 7f10ccff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.087 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.057+0000 7f10ccff9640 10 client.?.objecter ms_handle_connect 0x7f10d00585e0 2026-02-25T15:01:44.087 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.057+0000 7f10ccff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.087 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 140395313 0 0) 0x7f192c0ac290 con 0x7f192c157b50 2026-02-25T15:01:44.087 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.087 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.087 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.087 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 10 monclient(hunting): my global_id is 4575 2026-02-25T15:01:44.087 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18fc003610 con 0x7f192c157b50 2026-02-25T15:01:44.088 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f10ccff9640 10 client.?.objecter ms_handle_connect 0x7f10d00ba350 2026-02-25T15:01:44.088 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f10ccff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.088 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-25T15:01:44.089 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3498727863 0 0) 0x7f192c12b190 con 0x7f192c157780 2026-02-25T15:01:44.089 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 >> v1:10.20.193.94:6789/0 conn(0x7f192c164d00 legacy=0x7f192c15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.089 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 >> v1:10.20.193.90:6789/0 conn(0x7f192c157b50 legacy=0x7f192c0ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.089 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.089 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 1 monclient: found mon.b 2026-02-25T15:01:44.089 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.089 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.089 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192c12b470 con 0x7f192c157780 2026-02-25T15:01:44.089 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.089 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.089 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f1907fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.089 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f193209d640 5 monclient: authenticate success, global_id 4558 2026-02-25T15:01:44.089 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f193209d640 10 monclient: _renew_subs 2026-02-25T15:01:44.090 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f193209d640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.090 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.058+0000 7f193209d640 1 -- 10.20.193.90:0/2792121353 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f192c12b750 con 0x7f192c157780 2026-02-25T15:01:44.090 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.059+0000 7f193209d640 10 client.4558.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:44.090 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.059+0000 7f193209d640 10 monclient: _renew_subs 2026-02-25T15:01:44.090 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.059+0000 7f193209d640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.090 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.059+0000 7f193209d640 1 -- 10.20.193.90:0/2792121353 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f192c12ad80 con 0x7f192c157780 2026-02-25T15:01:44.090 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.059+0000 7f193209d640 1 librados: init done 2026-02-25T15:01:44.090 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.059+0000 7f193209d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.090 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.059+0000 7f193209d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.090 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.059+0000 7f193209d640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.090 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.059+0000 7f193209d640 1 -- 10.20.193.90:0/2792121353 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f192c0531e0 con 0x7f192c157780 2026-02-25T15:01:44.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.059+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3658646910 0 0) 0x7f10d015ee60 con 0x7f10d00585e0 2026-02-25T15:01:44.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.059+0000 7f10ccff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.059+0000 7f10ccff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.059+0000 7f10ccff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.059+0000 7f10ccff9640 10 monclient(hunting): my global_id is 4561 2026-02-25T15:01:44.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.059+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10a0003610 con 0x7f10d00585e0 2026-02-25T15:01:44.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.061+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3464206590 0 0) 0x7f10d015f140 con 0x7f10d00b9f80 2026-02-25T15:01:44.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.061+0000 7f10ccff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.061+0000 7f10ccff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.061+0000 7f10ccff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.061+0000 7f10ccff9640 10 monclient(hunting): my global_id is 4592 2026-02-25T15:01:44.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.061+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10d015ee60 con 0x7f10d00b9f80 2026-02-25T15:01:44.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.062+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4036752355 0 0) 0x7f10d015eb80 con 0x7f10d00ba350 2026-02-25T15:01:44.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.062+0000 7f10ccff9640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.062+0000 7f10ccff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.062+0000 7f10ccff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.062+0000 7f10ccff9640 10 monclient(hunting): my global_id is 4578 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.062+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10d015f140 con 0x7f10d00ba350 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.063+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 974849646 0 0) 0x7f10d015ee60 con 0x7f10d00b9f80 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.063+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f10d015eb80 con 0x7f10d00b9f80 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.063+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f10c0002f90 con 0x7f10d00b9f80 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.063+0000 7f10ccff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.063+0000 7f10ccff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.063+0000 7f10ccff9640 10 monclient(hunting): dump: 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.063+0000 7f10ccff9640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:44.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.1 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3411550258 0 0) 0x7f10d015eb80 con 0x7f10d00b9f80 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 >> v1:10.20.193.93:6789/0 conn(0x7f10d00585e0 legacy=0x7f10d0155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 >> v1:10.20.193.90:6789/0 conn(0x7f10d00ba350 legacy=0x7f10d01582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10ccff9640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10ccff9640 1 monclient: found mon.c 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10ccff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10ccff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d015f420 con 0x7f10d00b9f80 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10ccff9640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10ccff9640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10ccff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10d6488640 5 monclient: authenticate success, global_id 4592 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10d6488640 10 monclient: _renew_subs 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10d6488640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10d6488640 1 -- 10.20.193.90:0/235727452 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f10d015f700 con 0x7f10d00b9f80 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10d6488640 10 client.4592.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10d6488640 10 monclient: _renew_subs 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10d6488640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.094 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10d6488640 1 -- 10.20.193.90:0/235727452 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f10d015e8e0 con 0x7f10d00b9f80 2026-02-25T15:01:44.095 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.065+0000 7f10d6488640 1 librados: init done 2026-02-25T15:01:44.095 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.066+0000 7f10d6488640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.095 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.066+0000 7f10d6488640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.095 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.066+0000 7f10d6488640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.095 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.066+0000 7f10d6488640 1 -- 10.20.193.90:0/235727452 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d015b550 con 0x7f10d00b9f80 2026-02-25T15:01:44.095 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.070+0000 7f08dc31a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-25T15:01:44.095 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.070+0000 7f08dc31a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-25T15:01:44.095 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.070+0000 7f08dc31a640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.095 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.070+0000 7f08dc31a640 1 -- 10.20.193.90:0/600907781 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f08d4156740 con 0x7f08d4157b50 2026-02-25T15:01:44.097 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.087+0000 7fa17adb1640 10 monclient: get_monmap_and_config 2026-02-25T15:01:44.097 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.087+0000 7fa17adb1640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.097 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.087+0000 7fa17adb1640 10 monclient: monmap: 2026-02-25T15:01:44.097 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.097 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.097 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:44.088282+0000 2026-02-25T15:01:44.097 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:44.088282+0000 2026-02-25T15:01:44.098 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.098 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.098 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.098 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.098 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.098 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.088+0000 7fa17adb1640 1 Processor -- start 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.088+0000 7fa17adb1640 1 -- start start 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.088+0000 7fa17adb1640 10 monclient: init 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.088+0000 7fa17adb1640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.088+0000 7fa17adb1640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.088+0000 7fa17adb1640 10 monclient(hunting): picked mon.noname-b con 0x7fa174157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.088+0000 7fa17adb1640 10 monclient(hunting): picked mon.noname-a con 0x7fa174157b50 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.088+0000 7fa17adb1640 10 monclient(hunting): picked mon.noname-c con 0x7fa174164d00 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.088+0000 7fa17adb1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa17405de70 con 0x7fa174157b50 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.088+0000 7fa17adb1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa174055b20 con 0x7fa174157780 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.088+0000 7fa17adb1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa174055e00 con 0x7fa174164d00 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.088+0000 7fa17adb1640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.088+0000 7fa17adb1640 10 monclient(hunting): authenticate will time out at 2993.688607s 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.089+0000 7fa16bfff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fa174157b50 0x7fa1741645f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57272/0 (socket says 10.20.193.90:57272) 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.089+0000 7fa16bfff640 1 -- 10.20.193.90:0/962486401 learned_addr learned my addr 10.20.193.90:0/962486401 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.090+0000 7fa16b7fe640 1 -- 10.20.193.90:0/962486401 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3286108114 0 0) 0x7fa17405de70 con 0x7fa174157b50 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.090+0000 7fa16b7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.090+0000 7fa16b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.090+0000 7fa16b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.090+0000 7fa16b7fe640 10 monclient(hunting): my global_id is 4581 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.090+0000 7fa16b7fe640 1 -- 10.20.193.90:0/962486401 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa160003610 con 0x7fa174157b50 2026-02-25T15:01:44.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa16b7fe640 1 -- 10.20.193.90:0/962486401 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1529762697 0 0) 0x7fa160003610 con 0x7fa174157b50 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa16b7fe640 1 -- 10.20.193.90:0/962486401 >> v1:10.20.193.94:6789/0 conn(0x7fa174164d00 legacy=0x7fa1741c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa16b7fe640 1 -- 10.20.193.90:0/962486401 >> v1:10.20.193.93:6789/0 conn(0x7fa174157780 legacy=0x7fa174056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa16b7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa16b7fe640 1 monclient: found mon.noname-a 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa16b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa16b7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa16b7fe640 1 -- 10.20.193.90:0/962486401 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1740560e0 con 0x7fa174157b50 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa16b7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa16b7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa16b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa16b7fe640 1 -- 10.20.193.90:0/962486401 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa15c002df0 con 0x7fa174157b50 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa17adb1640 5 monclient: authenticate success, global_id 4581 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa17adb1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa16b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa16b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa16b7fe640 10 monclient: dump: 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.100 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.092+0000 7fa17adb1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.093+0000 7fa16b7fe640 1 -- 10.20.193.90:0/962486401 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa15c003000 con 0x7fa174157b50 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.093+0000 7fa16b7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.093+0000 7fa16b7fe640 1 -- 10.20.193.90:0/962486401 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa15c003440 con 0x7fa174157b50 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.093+0000 7fa16b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.093+0000 7fa16b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.093+0000 7fa16b7fe640 10 monclient: dump: 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.093+0000 7fa17adb1640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.093+0000 7fa17adb1640 4 set_mon_vals no callback set 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 10 monclient: shutdown 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 1 -- 10.20.193.90:0/962486401 >> v1:10.20.193.90:6789/0 conn(0x7fa174157b50 legacy=0x7fa1741645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 1 -- 10.20.193.90:0/962486401 shutdown_connections 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 1 -- 10.20.193.90:0/962486401 >> 10.20.193.90:0/962486401 conn(0x7fa1740b2ef0 msgr2=0x7fa1740b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 1 -- 10.20.193.90:0/962486401 shutdown_connections 2026-02-25T15:01:44.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 1 -- 10.20.193.90:0/962486401 wait complete. 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 10 monclient: monmap: 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 1 librados: starting msgr at 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 1 librados: starting objecter 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 1 Processor -- start 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 1 -- start start 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 1 librados: setting wanted keys 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 1 librados: calling monclient init 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.094+0000 7fa17adb1640 10 monclient: init 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.095+0000 7fa17adb1640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.095+0000 7fa17adb1640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.095+0000 7fa17adb1640 10 monclient(hunting): picked mon.noname-b con 0x7fa174157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.095+0000 7fa17adb1640 10 monclient(hunting): picked mon.noname-c con 0x7fa174157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.095+0000 7fa17adb1640 10 monclient(hunting): picked mon.noname-a con 0x7fa174164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.095+0000 7fa17adb1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa17415de30 con 0x7fa174164d00 2026-02-25T15:01:44.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.095+0000 7fa17adb1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa17415e110 con 0x7fa174157780 2026-02-25T15:01:44.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.095+0000 7fa17adb1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa17415e3f0 con 0x7fa174157b50 2026-02-25T15:01:44.104 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-25T15:01:44.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.095+0000 7fa17adb1640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.095+0000 7fa17adb1640 10 monclient(hunting): authenticate will time out at 2993.695247s 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.095+0000 7fa179327640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fa174164d00 0x7fa17415b8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57274/0 (socket says 10.20.193.90:57274) 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.095+0000 7fa179327640 1 -- 10.20.193.90:0/3869471588 learned_addr learned my addr 10.20.193.90:0/3869471588 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.095+0000 7fa1697fa640 10 client.?.objecter ms_handle_connect 0x7fa174164d00 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.095+0000 7fa1697fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.097+0000 7fa1697fa640 1 -- 10.20.193.90:0/3869471588 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1595252219 0 0) 0x7fa17415de30 con 0x7fa174164d00 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.097+0000 7fa1697fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.097+0000 7fa1697fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.097+0000 7fa1697fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.097+0000 7fa1697fa640 10 monclient(hunting): my global_id is 4584 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.097+0000 7fa1697fa640 1 -- 10.20.193.90:0/3869471588 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa144003610 con 0x7fa174164d00 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.100+0000 7fa1697fa640 1 -- 10.20.193.90:0/3869471588 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4022605188 0 0) 0x7fa144003610 con 0x7fa174164d00 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.100+0000 7fa1697fa640 1 -- 10.20.193.90:0/3869471588 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa17415de30 con 0x7fa174164d00 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.100+0000 7fa1697fa640 1 -- 10.20.193.90:0/3869471588 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa170003230 con 0x7fa174164d00 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.100+0000 7fa1697fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.100+0000 7fa1697fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.100+0000 7fa1697fa640 10 monclient(hunting): dump: 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.100+0000 7fa1697fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa1697fa640 1 -- 10.20.193.90:0/3869471588 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3125187319 0 0) 0x7fa17415de30 con 0x7fa174164d00 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa1697fa640 1 -- 10.20.193.90:0/3869471588 >> v1:10.20.193.94:6789/0 conn(0x7fa174157b50 legacy=0x7fa17405c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa1697fa640 1 -- 10.20.193.90:0/3869471588 >> v1:10.20.193.93:6789/0 conn(0x7fa174157780 legacy=0x7fa17412cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa1697fa640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa1697fa640 1 monclient: found mon.a 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa1697fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa1697fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa1697fa640 1 -- 10.20.193.90:0/3869471588 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17415e6d0 con 0x7fa174164d00 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa1697fa640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa1697fa640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa1697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa17adb1640 5 monclient: authenticate success, global_id 4584 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa17adb1640 10 monclient: _renew_subs 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa17adb1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa17adb1640 1 -- 10.20.193.90:0/3869471588 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa17412c890 con 0x7fa174164d00 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa17adb1640 10 client.4584.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa17adb1640 10 monclient: _renew_subs 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa17adb1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa17adb1640 1 -- 10.20.193.90:0/3869471588 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa17412c390 con 0x7fa174164d00 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.102+0000 7fa17adb1640 1 librados: init done 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.103+0000 7fa17adb1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.103+0000 7fa17adb1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.103+0000 7fa17adb1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.103+0000 7fa17adb1640 1 -- 10.20.193.90:0/3869471588 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1741647c0 con 0x7fa174164d00 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.151+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1918003520 con 0x7f192c157780 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.151+0000 7f1907fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.151+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f1918003b50 con 0x7f192c157780 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.151+0000 7f1907fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.151+0000 7f1907fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.151+0000 7f1907fff640 10 monclient: dump: 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.151+0000 7f1929ffb640 4 set_mon_vals no callback set 2026-02-25T15:01:44.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.152+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f1918003d30 con 0x7f192c157780 2026-02-25T15:01:44.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.153+0000 7f192affd640 10 monclient: get_auth_request con 0x7f18fc079fb0 auth_method 0 2026-02-25T15:01:44.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.153+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f1918095d40 con 0x7f192c157780 2026-02-25T15:01:44.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.153+0000 7f1907fff640 10 client.4558.objecter ms_dispatch2 0x7f192c000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:44.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.153+0000 7f1907fff640 3 client.4558.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:44.155 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.153+0000 7f1907fff640 3 client.4558.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:44.155 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.153+0000 7f1907fff640 20 client.4558.objecter dump_active .. 0 homeless 2026-02-25T15:01:44.155 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.153+0000 7f1907fff640 10 client.4558.objecter ms_handle_connect 0x7f18fc079fb0 2026-02-25T15:01:44.157 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.156+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f10c00048d0 con 0x7f10d00b9f80 2026-02-25T15:01:44.157 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.156+0000 7f10ccff9640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.157 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.156+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f10c0004ca0 con 0x7f10d00b9f80 2026-02-25T15:01:44.157 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.156+0000 7f10ccff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.157 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.156+0000 7f10ccff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:44.157 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.156+0000 7f10ccff9640 10 monclient: dump: 2026-02-25T15:01:44.157 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.157 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.157 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.157 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.157 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.157 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.157 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.157 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.158 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.158 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.156+0000 7f10ce7fc640 4 set_mon_vals no callback set 2026-02-25T15:01:44.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.156+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f192c0531e0 con 0x7f192c157780 2026-02-25T15:01:44.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.156+0000 7f1907fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.156+0000 7f1907fff640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:44.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.157+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f10c0004e80 con 0x7f10d00b9f80 2026-02-25T15:01:44.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.157+0000 7f10cffff640 10 monclient: get_auth_request con 0x7f10a0079c20 auth_method 0 2026-02-25T15:01:44.158 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.157+0000 7f10ccff9640 10 client.4592.objecter ms_handle_connect 0x7f10a0079c20 2026-02-25T15:01:44.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.158+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f10c005ad80 con 0x7f10d00b9f80 2026-02-25T15:01:44.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.158+0000 7f10ccff9640 10 client.4592.objecter ms_dispatch2 0x7f10d0000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:44.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.158+0000 7f10ccff9640 3 client.4592.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:44.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.158+0000 7f10ccff9640 3 client.4592.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:44.159 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.158+0000 7f10ccff9640 20 client.4592.objecter dump_active .. 0 homeless 2026-02-25T15:01:44.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.158+0000 7f08baffd640 1 -- 10.20.193.90:0/600907781 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (unknown 574334944 0 778662779) 0x7f08c805f6f0 con 0x7f08d4157b50 2026-02-25T15:01:44.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.158+0000 7f08baffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-25T15:01:44.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.158+0000 7f08baffd640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.160 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:44.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.159+0000 7f145d31a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-25T15:01:44.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.159+0000 7f145d31a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-25T15:01:44.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.159+0000 7f145d31a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.159+0000 7f145d31a640 1 -- 10.20.193.90:0/2140504661 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f14580525e0 con 0x7f14581fb2d0 2026-02-25T15:01:44.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.159+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f10c005f4f0 con 0x7f10d00b9f80 2026-02-25T15:01:44.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.159+0000 7f10ccff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.159+0000 7f10ccff9640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:44.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.162+0000 7f08dc31a640 10 librados: watch_flush enter 2026-02-25T15:01:44.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.162+0000 7f08dc31a640 10 librados: watch_flush exit 2026-02-25T15:01:44.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.162+0000 7f08dc31a640 20 client.4549.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.162+0000 7f08dc31a640 10 client.4549.objecter successfully canceled tick 2026-02-25T15:01:44.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.162+0000 7f08dc31a640 1 -- 10.20.193.90:0/600907781 >> v1:10.20.193.189:6800/3647375982 conn(0x7f08a4079e60 legacy=0x7f08a409a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.162+0000 7f08dc31a640 10 monclient: shutdown 2026-02-25T15:01:44.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.162+0000 7f08dc31a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.162+0000 7f08dc31a640 1 -- 10.20.193.90:0/600907781 >> v1:10.20.193.93:6789/0 conn(0x7f08d4157b50 legacy=0x7f08d40afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.163+0000 7f08dc31a640 1 -- 10.20.193.90:0/600907781 shutdown_connections 2026-02-25T15:01:44.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.163+0000 7f08dc31a640 1 -- 10.20.193.90:0/600907781 >> 10.20.193.90:0/600907781 conn(0x7f08d40b2ef0 msgr2=0x7f08d4059510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.163+0000 7f08dc31a640 1 -- 10.20.193.90:0/600907781 shutdown_connections 2026-02-25T15:01:44.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.163+0000 7f08dc31a640 1 -- 10.20.193.90:0/600907781 wait complete. 2026-02-25T15:01:44.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.163+0000 7f08dc31a640 1 librados: shutdown 2026-02-25T15:01:44.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.163+0000 7fd58ccaa640 10 monclient: get_monmap_and_config 2026-02-25T15:01:44.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.163+0000 7fd58ccaa640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.163+0000 7fd58ccaa640 10 monclient: monmap: 2026-02-25T15:01:44.166 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.166 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.166 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:44.164044+0000 2026-02-25T15:01:44.166 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:44.164044+0000 2026-02-25T15:01:44.166 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.166 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.166 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.166 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.166 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.166 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.163+0000 7fd58ccaa640 1 Processor -- start 2026-02-25T15:01:44.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.163+0000 7fd58ccaa640 1 -- start start 2026-02-25T15:01:44.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.163+0000 7fd58ccaa640 10 monclient: init 2026-02-25T15:01:44.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.164+0000 7fd58ccaa640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.164+0000 7fd58ccaa640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:44.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.164+0000 7fd58ccaa640 10 monclient(hunting): picked mon.noname-a con 0x7fd5880b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.164+0000 7fd58ccaa640 10 monclient(hunting): picked mon.noname-c con 0x7fd5880ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.164+0000 7fd58ccaa640 10 monclient(hunting): picked mon.noname-b con 0x7fd5880585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.164+0000 7fd58ccaa640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd588163930 con 0x7fd5880b9f80 2026-02-25T15:01:44.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.164+0000 7fd58ccaa640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd588163bb0 con 0x7fd5880585e0 2026-02-25T15:01:44.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.164+0000 7fd58ccaa640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd588163e90 con 0x7fd5880ba350 2026-02-25T15:01:44.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.164+0000 7fd58ccaa640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.164+0000 7fd58ccaa640 10 monclient(hunting): authenticate will time out at 2993.764553s 2026-02-25T15:01:44.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.164+0000 7fd586d76640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fd5880585e0 0x7fd588166cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:51068/0 (socket says 10.20.193.90:51068) 2026-02-25T15:01:44.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.164+0000 7fd586d76640 1 -- 10.20.193.90:0/2286719605 learned_addr learned my addr 10.20.193.90:0/2286719605 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.166+0000 7fd585573640 1 -- 10.20.193.90:0/2286719605 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3145653748 0 0) 0x7fd588163bb0 con 0x7fd5880585e0 2026-02-25T15:01:44.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.166+0000 7fd585573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.166+0000 7fd585573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.166+0000 7fd585573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.166+0000 7fd585573640 10 monclient(hunting): my global_id is 4564 2026-02-25T15:01:44.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.166+0000 7fd585573640 1 -- 10.20.193.90:0/2286719605 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd574003610 con 0x7fd5880585e0 2026-02-25T15:01:44.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.166+0000 7fd585573640 1 -- 10.20.193.90:0/2286719605 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1743876052 0 0) 0x7fd588163e90 con 0x7fd5880ba350 2026-02-25T15:01:44.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.166+0000 7fd585573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.166+0000 7fd585573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.166+0000 7fd585573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.166+0000 7fd585573640 10 monclient(hunting): my global_id is 4595 2026-02-25T15:01:44.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.166+0000 7fd585573640 1 -- 10.20.193.90:0/2286719605 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd588163bb0 con 0x7fd5880ba350 2026-02-25T15:01:44.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd585573640 1 -- 10.20.193.90:0/2286719605 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1097739720 0 0) 0x7fd574003610 con 0x7fd5880585e0 2026-02-25T15:01:44.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd585573640 1 -- 10.20.193.90:0/2286719605 >> v1:10.20.193.94:6789/0 conn(0x7fd5880ba350 legacy=0x7fd588057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd585573640 1 -- 10.20.193.90:0/2286719605 >> v1:10.20.193.90:6789/0 conn(0x7fd5880b9f80 legacy=0x7fd588163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:44.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd585573640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd585573640 1 monclient: found mon.noname-b 2026-02-25T15:01:44.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd585573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd585573640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.170 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd585573640 1 -- 10.20.193.90:0/2286719605 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5880573f0 con 0x7fd5880585e0 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd585573640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd585573640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd585573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd585573640 1 -- 10.20.193.90:0/2286719605 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd578003170 con 0x7fd5880585e0 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd585573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd585573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd585573640 10 monclient: dump: 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.171 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd58ccaa640 5 monclient: authenticate success, global_id 4564 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.167+0000 7fd58ccaa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.168+0000 7fd585573640 1 -- 10.20.193.90:0/2286719605 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd578003380 con 0x7fd5880585e0 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.168+0000 7fd585573640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.168+0000 7fd585573640 1 -- 10.20.193.90:0/2286719605 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd5780037c0 con 0x7fd5880585e0 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.168+0000 7fd585573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.168+0000 7fd585573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.168+0000 7fd585573640 10 monclient: dump: 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.172 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.173 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.168+0000 7fd58ccaa640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:44.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.168+0000 7fd58ccaa640 4 set_mon_vals no callback set 2026-02-25T15:01:44.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.169+0000 7fd58ccaa640 10 monclient: shutdown 2026-02-25T15:01:44.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.169+0000 7fd58ccaa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.169+0000 7fd58ccaa640 1 -- 10.20.193.90:0/2286719605 >> v1:10.20.193.93:6789/0 conn(0x7fd5880585e0 legacy=0x7fd588166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.169+0000 7fd58ccaa640 1 -- 10.20.193.90:0/2286719605 shutdown_connections 2026-02-25T15:01:44.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.169+0000 7fd58ccaa640 1 -- 10.20.193.90:0/2286719605 >> 10.20.193.90:0/2286719605 conn(0x7fd5880b7990 msgr2=0x7fd5880b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.169+0000 7fd58ccaa640 1 -- 10.20.193.90:0/2286719605 shutdown_connections 2026-02-25T15:01:44.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.169+0000 7fd58ccaa640 1 -- 10.20.193.90:0/2286719605 wait complete. 2026-02-25T15:01:44.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.169+0000 7fd58ccaa640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.169+0000 7fd58ccaa640 10 monclient: monmap: 2026-02-25T15:01:44.174 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.175 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:44.175 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:44.175 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:44.175 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.175 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:44.175 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.175 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.175 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.175 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 1 librados: starting msgr at 2026-02-25T15:01:44.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 1 librados: starting objecter 2026-02-25T15:01:44.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:44.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 1 Processor -- start 2026-02-25T15:01:44.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 1 -- start start 2026-02-25T15:01:44.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 1 librados: setting wanted keys 2026-02-25T15:01:44.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 1 librados: calling monclient init 2026-02-25T15:01:44.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 10 monclient: init 2026-02-25T15:01:44.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:44.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 10 monclient(hunting): picked mon.noname-a con 0x7fd5880585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 10 monclient(hunting): picked mon.noname-b con 0x7fd5880b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 10 monclient(hunting): picked mon.noname-c con 0x7fd5880ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5880aefa0 con 0x7fd5880585e0 2026-02-25T15:01:44.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5880b11e0 con 0x7fd5880b9f80 2026-02-25T15:01:44.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5880b14c0 con 0x7fd5880ba350 2026-02-25T15:01:44.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7fd58ccaa640 10 monclient(hunting): authenticate will time out at 2993.770665s 2026-02-25T15:01:44.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7ff74858a640 10 monclient: get_monmap_and_config 2026-02-25T15:01:44.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7ff74858a640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.170+0000 7ff74858a640 10 monclient: monmap: 2026-02-25T15:01:44.177 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.177 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.177 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:44.171440+0000 2026-02-25T15:01:44.177 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:44.171440+0000 2026-02-25T15:01:44.177 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.177 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.177 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.177 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.177 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.177 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7fd586d76640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fd5880ba350 0x7fd588161800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45718/0 (socket says 10.20.193.90:45718) 2026-02-25T15:01:44.177 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7fd586d76640 1 -- 10.20.193.90:0/2898396181 learned_addr learned my addr 10.20.193.90:0/2898396181 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7ff74858a640 1 Processor -- start 2026-02-25T15:01:44.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7ff74858a640 1 -- start start 2026-02-25T15:01:44.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7ff74858a640 10 monclient: init 2026-02-25T15:01:44.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7fd56effd640 10 client.?.objecter ms_handle_connect 0x7fd5880ba350 2026-02-25T15:01:44.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7fd56effd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7fd56effd640 10 client.?.objecter ms_handle_connect 0x7fd5880b9f80 2026-02-25T15:01:44.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7fd56effd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7ff74858a640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7ff74858a640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:44.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7ff74858a640 10 monclient(hunting): picked mon.noname-c con 0x7ff740159da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7ff74858a640 10 monclient(hunting): picked mon.noname-a con 0x7ff740165880 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7ff74858a640 10 monclient(hunting): picked mon.noname-b con 0x7ff740056090 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7ff74858a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff74015a170 con 0x7ff740165880 2026-02-25T15:01:44.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7ff74858a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7400b4650 con 0x7ff740056090 2026-02-25T15:01:44.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7ff74858a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7400b4930 con 0x7ff740159da0 2026-02-25T15:01:44.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7ff74858a640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.180 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.171+0000 7ff74858a640 10 monclient(hunting): authenticate will time out at 2993.771779s 2026-02-25T15:01:44.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.0 2026-02-25T15:01:44.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.172+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2219253996 0 0) 0x7fd5880b14c0 con 0x7fd5880ba350 2026-02-25T15:01:44.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.172+0000 7fd56effd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.172+0000 7fd56effd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.172+0000 7fd56effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.172+0000 7fd56effd640 10 monclient(hunting): my global_id is 4598 2026-02-25T15:01:44.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.172+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd554003610 con 0x7fd5880ba350 2026-02-25T15:01:44.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.172+0000 7ff7462ff640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7ff740159da0 0x7ff740165170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45730/0 (socket says 10.20.193.90:45730) 2026-02-25T15:01:44.181 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.172+0000 7ff7462ff640 1 -- 10.20.193.90:0/1752047464 learned_addr learned my addr 10.20.193.90:0/1752047464 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.172+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 596269494 0 0) 0x7fd5880b11e0 con 0x7fd5880b9f80 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.172+0000 7fd56effd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.172+0000 7fd56effd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.172+0000 7fd56effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.172+0000 7fd56effd640 10 monclient(hunting): my global_id is 4567 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.173+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5880b14c0 con 0x7fd5880b9f80 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.175+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3933579890 0 0) 0x7fd554003610 con 0x7fd5880ba350 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.175+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd5880b11e0 con 0x7fd5880ba350 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.175+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd5780036d0 con 0x7fd5880ba350 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.175+0000 7fd56effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.175+0000 7fd56effd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.175+0000 7fd56effd640 10 monclient(hunting): dump: 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.182 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.183 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.183 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.183 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.183 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.183 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.175+0000 7fd56effd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:44.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.175+0000 7ff7452fd640 1 -- 10.20.193.90:0/1752047464 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3999165535 0 0) 0x7ff7400b4930 con 0x7ff740159da0 2026-02-25T15:01:44.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.175+0000 7ff7452fd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.175+0000 7ff7452fd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.175+0000 7ff7452fd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.175+0000 7ff7452fd640 10 monclient(hunting): my global_id is 4601 2026-02-25T15:01:44.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.175+0000 7ff7452fd640 1 -- 10.20.193.90:0/1752047464 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff728003610 con 0x7ff740159da0 2026-02-25T15:01:44.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.176+0000 7ff7452fd640 1 -- 10.20.193.90:0/1752047464 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2688247618 0 0) 0x7ff7400b4650 con 0x7ff740056090 2026-02-25T15:01:44.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.176+0000 7ff7452fd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.176+0000 7ff7452fd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.176+0000 7ff7452fd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.176+0000 7ff7452fd640 10 monclient(hunting): my global_id is 4570 2026-02-25T15:01:44.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.176+0000 7ff7452fd640 1 -- 10.20.193.90:0/1752047464 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7400b4930 con 0x7ff740056090 2026-02-25T15:01:44.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.176+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 708518508 0 0) 0x7fd5880b14c0 con 0x7fd5880b9f80 2026-02-25T15:01:44.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.176+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd554003610 con 0x7fd5880b9f80 2026-02-25T15:01:44.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.176+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd570002fb0 con 0x7fd5880b9f80 2026-02-25T15:01:44.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.176+0000 7fd56effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.176+0000 7fd56effd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:44.185 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.176+0000 7fd56effd640 10 monclient(hunting): dump: 2026-02-25T15:01:44.185 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.185 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.185 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.185 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.185 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.185 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.185 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.185 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.185 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.177+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3348909383 0 0) 0x7fd5880b11e0 con 0x7fd5880ba350 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.177+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 >> v1:10.20.193.93:6789/0 conn(0x7fd5880b9f80 legacy=0x7fd5880b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.177+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 >> v1:10.20.193.90:6789/0 conn(0x7fd5880585e0 legacy=0x7fd5880ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.177+0000 7fd56effd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.177+0000 7fd56effd640 1 monclient: found mon.c 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.177+0000 7fd56effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.177+0000 7fd56effd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.177+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5880b17a0 con 0x7fd5880ba350 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.177+0000 7fd56effd640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.177+0000 7fd56effd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.177+0000 7fd56effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.177+0000 7fd58ccaa640 5 monclient: authenticate success, global_id 4598 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.177+0000 7fd58ccaa640 10 monclient: _renew_subs 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.177+0000 7fd58ccaa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.177+0000 7fd58ccaa640 1 -- 10.20.193.90:0/2898396181 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd588161000 con 0x7fd5880ba350 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.178+0000 7fd58ccaa640 10 client.4598.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.178+0000 7fd58ccaa640 10 monclient: _renew_subs 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.178+0000 7fd58ccaa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.178+0000 7fd58ccaa640 1 -- 10.20.193.90:0/2898396181 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd5880b0e80 con 0x7fd5880ba350 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.178+0000 7fd58ccaa640 1 librados: init done 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.178+0000 7fd58ccaa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.186 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.178+0000 7fd58ccaa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.178+0000 7fd58ccaa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.178+0000 7fd58ccaa640 1 -- 10.20.193.90:0/2898396181 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd588055260 con 0x7fd5880ba350 2026-02-25T15:01:44.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff7452fd640 1 -- 10.20.193.90:0/1752047464 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4041082310 0 0) 0x7ff728003610 con 0x7ff740159da0 2026-02-25T15:01:44.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff7452fd640 1 -- 10.20.193.90:0/1752047464 >> v1:10.20.193.93:6789/0 conn(0x7ff740056090 legacy=0x7ff7400b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff7452fd640 1 -- 10.20.193.90:0/1752047464 >> v1:10.20.193.90:6789/0 conn(0x7ff740165880 legacy=0x7ff740055980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:44.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff7452fd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff7452fd640 1 monclient: found mon.noname-c 2026-02-25T15:01:44.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff7452fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff7452fd640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff7452fd640 1 -- 10.20.193.90:0/1752047464 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7400b4c10 con 0x7ff740159da0 2026-02-25T15:01:44.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff7452fd640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff7452fd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff7452fd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.187 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff7452fd640 1 -- 10.20.193.90:0/1752047464 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff730002df0 con 0x7ff740159da0 2026-02-25T15:01:44.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff7452fd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff7452fd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:44.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff7452fd640 10 monclient: dump: 2026-02-25T15:01:44.188 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.188 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.188 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.188 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.188 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.188 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.188 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.188 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.188 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.188 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff74858a640 5 monclient: authenticate success, global_id 4601 2026-02-25T15:01:44.188 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.179+0000 7ff74858a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.196+0000 7fa1697fa640 1 -- 10.20.193.90:0/3869471588 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa1700038b0 con 0x7fa174164d00 2026-02-25T15:01:44.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.196+0000 7fa1697fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.196+0000 7fa1697fa640 1 -- 10.20.193.90:0/3869471588 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa170004e70 con 0x7fa174164d00 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.196+0000 7fa1697fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.196+0000 7fa1697fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.196+0000 7fa1697fa640 10 monclient: dump: 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.196+0000 7fa16affd640 4 set_mon_vals no callback set 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.197+0000 7fa1697fa640 1 -- 10.20.193.90:0/3869471588 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fa17001e350 con 0x7fa174164d00 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.197+0000 7fa178b26640 10 monclient: get_auth_request con 0x7fa144079e10 auth_method 0 2026-02-25T15:01:44.198 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.198+0000 7fa1697fa640 10 client.4584.objecter ms_handle_connect 0x7fa144079e10 2026-02-25T15:01:44.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.201+0000 7fa1697fa640 1 -- 10.20.193.90:0/3869471588 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7fa170095f90 con 0x7fa174164d00 2026-02-25T15:01:44.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.201+0000 7fa1697fa640 10 client.4584.objecter ms_dispatch2 0x7fa174000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:44.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.201+0000 7fa1697fa640 3 client.4584.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:44.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.201+0000 7fa1697fa640 3 client.4584.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:44.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.201+0000 7fa1697fa640 20 client.4584.objecter dump_active .. 0 homeless 2026-02-25T15:01:44.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.201+0000 7fa1697fa640 1 -- 10.20.193.90:0/3869471588 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fa17005f8f0 con 0x7fa174164d00 2026-02-25T15:01:44.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.201+0000 7fa1697fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.202 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.201+0000 7fa1697fa640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:44.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.209+0000 7f14377fe640 1 -- 10.20.193.90:0/2140504661 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+12 (unknown 3207554136 0 778662779) 0x7f144805f8a0 con 0x7f14581fb2d0 2026-02-25T15:01:44.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.209+0000 7f14377fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-25T15:01:44.210 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.209+0000 7f14377fe640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.210 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:44.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.212+0000 7f145d31a640 10 librados: watch_flush enter 2026-02-25T15:01:44.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.212+0000 7f145d31a640 10 librados: watch_flush exit 2026-02-25T15:01:44.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.212+0000 7f145d31a640 20 client.4572.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.212+0000 7f145d31a640 10 client.4572.objecter successfully canceled tick 2026-02-25T15:01:44.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.212+0000 7f145d31a640 1 -- 10.20.193.90:0/2140504661 >> v1:10.20.193.189:6800/3647375982 conn(0x7f1424079e90 legacy=0x7f142409a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.212+0000 7f145d31a640 10 monclient: shutdown 2026-02-25T15:01:44.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.212+0000 7f145d31a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.212+0000 7f145d31a640 1 -- 10.20.193.90:0/2140504661 >> v1:10.20.193.90:6789/0 conn(0x7f14581fb2d0 legacy=0x7f1458135890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.212+0000 7f145d31a640 1 -- 10.20.193.90:0/2140504661 shutdown_connections 2026-02-25T15:01:44.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.212+0000 7f145d31a640 1 -- 10.20.193.90:0/2140504661 >> 10.20.193.90:0/2140504661 conn(0x7f14580b5290 msgr2=0x7f14580b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.212+0000 7f145d31a640 1 -- 10.20.193.90:0/2140504661 shutdown_connections 2026-02-25T15:01:44.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.212+0000 7f145d31a640 1 -- 10.20.193.90:0/2140504661 wait complete. 2026-02-25T15:01:44.213 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.212+0000 7f145d31a640 1 librados: shutdown 2026-02-25T15:01:44.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.8 2026-02-25T15:01:44.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.266+0000 7f193209d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-25T15:01:44.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.266+0000 7f193209d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-25T15:01:44.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.266+0000 7f193209d640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.266+0000 7f193209d640 1 -- 10.20.193.90:0/2792121353 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f192c05e250 con 0x7f192c157780 2026-02-25T15:01:44.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.267+0000 7f10d6488640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-25T15:01:44.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.267+0000 7f10d6488640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-25T15:01:44.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.267+0000 7f10d6488640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.267+0000 7f10d6488640 1 -- 10.20.193.90:0/235727452 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f10d015b270 con 0x7f10d00b9f80 2026-02-25T15:01:44.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.268+0000 7f1907fff640 1 -- 10.20.193.90:0/2792121353 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+12 (unknown 2894986287 0 778662779) 0x7f192c05e250 con 0x7f192c157780 2026-02-25T15:01:44.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.268+0000 7f1907fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-25T15:01:44.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.268+0000 7f1907fff640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.270 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:44.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.270+0000 7f193209d640 10 librados: watch_flush enter 2026-02-25T15:01:44.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.270+0000 7f193209d640 10 librados: watch_flush exit 2026-02-25T15:01:44.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.270+0000 7f193209d640 20 client.4558.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.270+0000 7f193209d640 10 client.4558.objecter successfully canceled tick 2026-02-25T15:01:44.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.270+0000 7f193209d640 1 -- 10.20.193.90:0/2792121353 >> v1:10.20.193.189:6800/3647375982 conn(0x7f18fc079fb0 legacy=0x7f18fc09a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.270+0000 7f193209d640 10 monclient: shutdown 2026-02-25T15:01:44.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.270+0000 7f193209d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.270+0000 7f193209d640 1 -- 10.20.193.90:0/2792121353 >> v1:10.20.193.93:6789/0 conn(0x7f192c157780 legacy=0x7f192c0afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.271+0000 7f193209d640 1 -- 10.20.193.90:0/2792121353 shutdown_connections 2026-02-25T15:01:44.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.271+0000 7f193209d640 1 -- 10.20.193.90:0/2792121353 >> 10.20.193.90:0/2792121353 conn(0x7f192c0b2ef0 msgr2=0x7f192c15fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.271+0000 7f193209d640 1 -- 10.20.193.90:0/2792121353 shutdown_connections 2026-02-25T15:01:44.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.271+0000 7f193209d640 1 -- 10.20.193.90:0/2792121353 wait complete. 2026-02-25T15:01:44.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.271+0000 7f193209d640 1 librados: shutdown 2026-02-25T15:01:44.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.272+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd578003380 con 0x7fd5880ba350 2026-02-25T15:01:44.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.272+0000 7fd56effd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.272+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd578005200 con 0x7fd5880ba350 2026-02-25T15:01:44.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.272+0000 7fd56effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.272+0000 7fd584d72640 4 set_mon_vals no callback set 2026-02-25T15:01:44.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.272+0000 7fd56effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:44.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.272+0000 7fd56effd640 10 monclient: dump: 2026-02-25T15:01:44.275 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.275 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.275 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.275 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.275 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.275 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.275 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.275 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.275 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.275 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.272+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fd57801e6e0 con 0x7fd5880ba350 2026-02-25T15:01:44.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.273+0000 7fd586575640 10 monclient: get_auth_request con 0x7fd554079ea0 auth_method 0 2026-02-25T15:01:44.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.273+0000 7fd56effd640 10 client.4598.objecter ms_handle_connect 0x7fd554079ea0 2026-02-25T15:01:44.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.274+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7fd578095f20 con 0x7fd5880ba350 2026-02-25T15:01:44.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.274+0000 7fd56effd640 10 client.4598.objecter ms_dispatch2 0x7fd588000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:44.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.274+0000 7fd56effd640 3 client.4598.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:44.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.274+0000 7fd56effd640 3 client.4598.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:44.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.274+0000 7fd56effd640 20 client.4598.objecter dump_active .. 0 homeless 2026-02-25T15:01:44.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.278+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fd5780cfd10 con 0x7fd5880ba350 2026-02-25T15:01:44.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.278+0000 7fd56effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.278+0000 7fd56effd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:44.279 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2026-02-25T15:01:44.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.278+0000 7f10ccff9640 1 -- 10.20.193.90:0/235727452 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+12 (unknown 3646472635 0 778662779) 0x7f10c005edf0 con 0x7f10d00b9f80 2026-02-25T15:01:44.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.278+0000 7f10ccff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-25T15:01:44.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.278+0000 7f10ccff9640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.280 INFO:teuthology.orchestra.run.trial090.stdout:90194313219 2026-02-25T15:01:44.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff7452fd640 1 -- 10.20.193.90:0/1752047464 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff730003000 con 0x7ff740159da0 2026-02-25T15:01:44.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff7452fd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff7452fd640 1 -- 10.20.193.90:0/1752047464 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff730003440 con 0x7ff740159da0 2026-02-25T15:01:44.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff7452fd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff7452fd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:44.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff7452fd640 10 monclient: dump: 2026-02-25T15:01:44.282 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.282 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.282 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.282 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.282 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.283 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.283 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.283 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.283 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.283 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff74858a640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:44.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff74858a640 4 set_mon_vals no callback set 2026-02-25T15:01:44.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff74858a640 10 monclient: shutdown 2026-02-25T15:01:44.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff74858a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff74858a640 1 -- 10.20.193.90:0/1752047464 >> v1:10.20.193.94:6789/0 conn(0x7ff740159da0 legacy=0x7ff740165170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff74858a640 1 -- 10.20.193.90:0/1752047464 shutdown_connections 2026-02-25T15:01:44.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff74858a640 1 -- 10.20.193.90:0/1752047464 >> 10.20.193.90:0/1752047464 conn(0x7ff7400b78c0 msgr2=0x7ff7400b7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff74858a640 1 -- 10.20.193.90:0/1752047464 shutdown_connections 2026-02-25T15:01:44.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff74858a640 1 -- 10.20.193.90:0/1752047464 wait complete. 2026-02-25T15:01:44.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.9 2026-02-25T15:01:44.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff74858a640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.279+0000 7ff74858a640 10 monclient: monmap: 2026-02-25T15:01:44.286 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.286 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:44.286 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:44.286 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:44.287 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.287 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:44.287 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.287 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.287 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.287 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 1 librados: starting msgr at 2026-02-25T15:01:44.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 1 librados: starting objecter 2026-02-25T15:01:44.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:44.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 1 Processor -- start 2026-02-25T15:01:44.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 1 -- start start 2026-02-25T15:01:44.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 1 librados: setting wanted keys 2026-02-25T15:01:44.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 1 librados: calling monclient init 2026-02-25T15:01:44.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 10 monclient: init 2026-02-25T15:01:44.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:44.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 10 monclient(hunting): picked mon.noname-a con 0x7ff740056090 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 10 monclient(hunting): picked mon.noname-b con 0x7ff740159da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 10 monclient(hunting): picked mon.noname-c con 0x7ff740165880 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7400b8ee0 con 0x7ff740056090 2026-02-25T15:01:44.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7400b91c0 con 0x7ff740159da0 2026-02-25T15:01:44.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7400b94a0 con 0x7ff740165880 2026-02-25T15:01:44.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff74858a640 10 monclient(hunting): authenticate will time out at 2993.880569s 2026-02-25T15:01:44.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff745afe640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7ff740159da0 0x7ff740137ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:51088/0 (socket says 10.20.193.90:51088) 2026-02-25T15:01:44.288 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.280+0000 7ff745afe640 1 -- 10.20.193.90:0/54761343 learned_addr learned my addr 10.20.193.90:0/54761343 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.281+0000 7ff726ffd640 10 client.?.objecter ms_handle_connect 0x7ff740159da0 2026-02-25T15:01:44.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.281+0000 7ff726ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.281+0000 7f10d6488640 10 librados: watch_flush enter 2026-02-25T15:01:44.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.281+0000 7f10d6488640 10 librados: watch_flush exit 2026-02-25T15:01:44.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.281+0000 7f10d6488640 20 client.4592.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.281+0000 7f10d6488640 10 client.4592.objecter successfully canceled tick 2026-02-25T15:01:44.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.281+0000 7f10d6488640 1 -- 10.20.193.90:0/235727452 >> v1:10.20.193.189:6800/3647375982 conn(0x7f10a0079c20 legacy=0x7f10a009a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.281+0000 7f10d6488640 10 monclient: shutdown 2026-02-25T15:01:44.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.281+0000 7f10d6488640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.281+0000 7f10d6488640 1 -- 10.20.193.90:0/235727452 >> v1:10.20.193.94:6789/0 conn(0x7f10d00b9f80 legacy=0x7f10d01fc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.281+0000 7f10d6488640 1 -- 10.20.193.90:0/235727452 shutdown_connections 2026-02-25T15:01:44.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.281+0000 7f10d6488640 1 -- 10.20.193.90:0/235727452 >> 10.20.193.90:0/235727452 conn(0x7f10d00b7990 msgr2=0x7f10d0157970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.281+0000 7f10d6488640 1 -- 10.20.193.90:0/235727452 shutdown_connections 2026-02-25T15:01:44.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.281+0000 7f10d6488640 1 -- 10.20.193.90:0/235727452 wait complete. 2026-02-25T15:01:44.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.281+0000 7f10d6488640 1 librados: shutdown 2026-02-25T15:01:44.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.282+0000 7ff726ffd640 10 client.?.objecter ms_handle_connect 0x7ff740165880 2026-02-25T15:01:44.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.282+0000 7ff726ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.292 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.10 2026-02-25T15:01:44.294 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-25T15:01:44.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.306+0000 7ff726ffd640 10 client.?.objecter ms_handle_connect 0x7ff740056090 2026-02-25T15:01:44.307 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.306+0000 7ff726ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.309 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2026-02-25T15:01:44.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.308+0000 7ff726ffd640 1 -- 10.20.193.90:0/54761343 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2636271375 0 0) 0x7ff7400b8ee0 con 0x7ff740056090 2026-02-25T15:01:44.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.308+0000 7ff726ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.310 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.308+0000 7ff726ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.308+0000 7ff726ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.308+0000 7ff726ffd640 10 monclient(hunting): my global_id is 4587 2026-02-25T15:01:44.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.308+0000 7ff726ffd640 1 -- 10.20.193.90:0/54761343 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff710003610 con 0x7ff740056090 2026-02-25T15:01:44.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.309+0000 7fa17adb1640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-25T15:01:44.311 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.309+0000 7fa17adb1640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-25T15:01:44.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.309+0000 7fa17adb1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.312 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.309+0000 7fa17adb1640 1 -- 10.20.193.90:0/3869471588 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 12} v 0) -- 0x7fa1741644e0 con 0x7fa174164d00 2026-02-25T15:01:44.313 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-25T15:01:44.314 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-25T15:01:44.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.312+0000 7ff726ffd640 1 -- 10.20.193.90:0/54761343 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1930529528 0 0) 0x7ff710003610 con 0x7ff740056090 2026-02-25T15:01:44.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.312+0000 7ff726ffd640 1 -- 10.20.193.90:0/54761343 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff7400b8ee0 con 0x7ff740056090 2026-02-25T15:01:44.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.312+0000 7ff726ffd640 1 -- 10.20.193.90:0/54761343 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff730003350 con 0x7ff740056090 2026-02-25T15:01:44.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.312+0000 7ff726ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.312+0000 7ff726ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:44.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.312+0000 7ff726ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:44.315 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.316 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.316 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.316 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.316 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.316 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.316 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.316 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.316 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.316 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.312+0000 7ff726ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:44.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.314+0000 7fa1697fa640 1 -- 10.20.193.90:0/3869471588 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 12}]=0 v0) ==== 75+0+12 (unknown 4263375189 0 1498640560) 0x7fa17005f1f0 con 0x7fa174164d00 2026-02-25T15:01:44.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.314+0000 7fa1697fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-25T15:01:44.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.314+0000 7fa1697fa640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.317 INFO:teuthology.orchestra.run.trial090.stdout:90194313221 2026-02-25T15:01:44.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.316+0000 7ff726ffd640 1 -- 10.20.193.90:0/54761343 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2537119435 0 0) 0x7ff7400b8ee0 con 0x7ff740056090 2026-02-25T15:01:44.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.316+0000 7ff726ffd640 1 -- 10.20.193.90:0/54761343 >> v1:10.20.193.94:6789/0 conn(0x7ff740165880 legacy=0x7ff7401584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.316+0000 7ff726ffd640 1 -- 10.20.193.90:0/54761343 >> v1:10.20.193.93:6789/0 conn(0x7ff740159da0 legacy=0x7ff740137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.316+0000 7ff726ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.316+0000 7ff726ffd640 1 monclient: found mon.a 2026-02-25T15:01:44.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.316+0000 7ff726ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.316+0000 7ff726ffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.316+0000 7ff726ffd640 1 -- 10.20.193.90:0/54761343 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7400b9780 con 0x7ff740056090 2026-02-25T15:01:44.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.316+0000 7ff726ffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.316+0000 7ff726ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.316+0000 7ff726ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.316+0000 7ff74858a640 5 monclient: authenticate success, global_id 4587 2026-02-25T15:01:44.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.316+0000 7ff74858a640 10 monclient: _renew_subs 2026-02-25T15:01:44.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.316+0000 7ff74858a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.316+0000 7ff74858a640 1 -- 10.20.193.90:0/54761343 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff7400b9a60 con 0x7ff740056090 2026-02-25T15:01:44.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7ff74858a640 10 client.4587.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:44.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7ff74858a640 10 monclient: _renew_subs 2026-02-25T15:01:44.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7ff74858a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7ff74858a640 1 -- 10.20.193.90:0/54761343 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff7400b8b60 con 0x7ff740056090 2026-02-25T15:01:44.320 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7ff74858a640 1 librados: init done 2026-02-25T15:01:44.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7ff74858a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7ff74858a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7ff74858a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7ff74858a640 1 -- 10.20.193.90:0/54761343 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7400565c0 con 0x7ff740056090 2026-02-25T15:01:44.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7fa17adb1640 10 librados: watch_flush enter 2026-02-25T15:01:44.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7fa17adb1640 10 librados: watch_flush exit 2026-02-25T15:01:44.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7fa17adb1640 20 client.4584.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7fa17adb1640 10 client.4584.objecter successfully canceled tick 2026-02-25T15:01:44.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7fa17adb1640 1 -- 10.20.193.90:0/3869471588 >> v1:10.20.193.189:6800/3647375982 conn(0x7fa144079e10 legacy=0x7fa14409a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7fa17adb1640 10 monclient: shutdown 2026-02-25T15:01:44.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7fa17adb1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7fa17adb1640 1 -- 10.20.193.90:0/3869471588 >> v1:10.20.193.90:6789/0 conn(0x7fa174164d00 legacy=0x7fa17415b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7fa17adb1640 1 -- 10.20.193.90:0/3869471588 shutdown_connections 2026-02-25T15:01:44.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7fa17adb1640 1 -- 10.20.193.90:0/3869471588 >> 10.20.193.90:0/3869471588 conn(0x7fa1740b2ef0 msgr2=0x7fa174056ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7fa17adb1640 1 -- 10.20.193.90:0/3869471588 shutdown_connections 2026-02-25T15:01:44.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7fa17adb1640 1 -- 10.20.193.90:0/3869471588 wait complete. 2026-02-25T15:01:44.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.317+0000 7fa17adb1640 1 librados: shutdown 2026-02-25T15:01:44.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.12 2026-02-25T15:01:44.328 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:44.329 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.326+0000 7ff726ffd640 1 -- 10.20.193.90:0/54761343 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff730003000 con 0x7ff740056090 2026-02-25T15:01:44.329 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.326+0000 7ff726ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.329 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.326+0000 7ff726ffd640 1 -- 10.20.193.90:0/54761343 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff730004e80 con 0x7ff740056090 2026-02-25T15:01:44.330 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.326+0000 7ff726ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.330 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.326+0000 7ff726ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:44.330 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.326+0000 7ff726ffd640 10 monclient: dump: 2026-02-25T15:01:44.330 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.330 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.330 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.330 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.330 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.330 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.330 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.330 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.331 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.331 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.331 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.326+0000 7ff744afc640 4 set_mon_vals no callback set 2026-02-25T15:01:44.331 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.327+0000 7ff726ffd640 1 -- 10.20.193.90:0/54761343 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7ff73001e360 con 0x7ff740056090 2026-02-25T15:01:44.331 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.328+0000 7ff745afe640 10 monclient: get_auth_request con 0x7ff710079f60 auth_method 0 2026-02-25T15:01:44.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.328+0000 7ff726ffd640 10 client.4587.objecter ms_handle_connect 0x7ff710079f60 2026-02-25T15:01:44.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.329+0000 7ff726ffd640 1 -- 10.20.193.90:0/54761343 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7ff730095be0 con 0x7ff740056090 2026-02-25T15:01:44.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.329+0000 7ff726ffd640 10 client.4587.objecter ms_dispatch2 0x7ff740000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:44.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.329+0000 7ff726ffd640 3 client.4587.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:44.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.329+0000 7ff726ffd640 3 client.4587.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:44.332 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.330+0000 7ff726ffd640 20 client.4587.objecter dump_active .. 0 homeless 2026-02-25T15:01:44.349 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.348+0000 7fab44144640 10 monclient: get_monmap_and_config 2026-02-25T15:01:44.349 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.348+0000 7fab44144640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.349 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.348+0000 7fab44144640 10 monclient: monmap: 2026-02-25T15:01:44.349 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.350 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.350 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:44.349015+0000 2026-02-25T15:01:44.350 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:44.349015+0000 2026-02-25T15:01:44.350 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.350 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.350 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.350 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.350 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.350 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.348+0000 7fab44144640 1 Processor -- start 2026-02-25T15:01:44.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.348+0000 7fab44144640 1 -- start start 2026-02-25T15:01:44.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.348+0000 7fab44144640 10 monclient: init 2026-02-25T15:01:44.352 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.349+0000 7fab44144640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.349+0000 7fab44144640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:44.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.349+0000 7fab44144640 10 monclient(hunting): picked mon.noname-a con 0x7fab3c0b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.349+0000 7fab44144640 10 monclient(hunting): picked mon.noname-c con 0x7fab3c0ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.349+0000 7fab44144640 10 monclient(hunting): picked mon.noname-b con 0x7fab3c0585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.349+0000 7fab44144640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c163930 con 0x7fab3c0b9f80 2026-02-25T15:01:44.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.349+0000 7fab44144640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c163bb0 con 0x7fab3c0585e0 2026-02-25T15:01:44.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.349+0000 7fab44144640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c163e90 con 0x7fab3c0ba350 2026-02-25T15:01:44.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.349+0000 7fab44144640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.349+0000 7fab44144640 10 monclient(hunting): authenticate will time out at 2993.949409s 2026-02-25T15:01:44.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.349+0000 7fab416b8640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fab3c0ba350 0x7fab3c057ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45746/0 (socket says 10.20.193.90:45746) 2026-02-25T15:01:44.353 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.349+0000 7fab416b8640 1 -- 10.20.193.90:0/4177226885 learned_addr learned my addr 10.20.193.90:0/4177226885 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.376+0000 7fe7e6c3a640 10 monclient: get_monmap_and_config 2026-02-25T15:01:44.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.376+0000 7fe7e6c3a640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.376+0000 7fe7e6c3a640 10 monclient: monmap: 2026-02-25T15:01:44.378 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.378 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.378 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:44.377641+0000 2026-02-25T15:01:44.378 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:44.377641+0000 2026-02-25T15:01:44.379 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.379 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.379 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.379 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.379 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.379 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.380 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.377+0000 7fe7e6c3a640 1 Processor -- start 2026-02-25T15:01:44.380 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.377+0000 7fe7e6c3a640 1 -- start start 2026-02-25T15:01:44.380 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.377+0000 7fe7e6c3a640 10 monclient: init 2026-02-25T15:01:44.380 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.377+0000 7fa4b3c16640 10 monclient: get_monmap_and_config 2026-02-25T15:01:44.380 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.377+0000 7fa4b3c16640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.380 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.377+0000 7fa4b3c16640 10 monclient: monmap: 2026-02-25T15:01:44.380 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.381 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.381 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:44.378317+0000 2026-02-25T15:01:44.381 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:44.378317+0000 2026-02-25T15:01:44.381 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.381 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.381 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.381 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.381 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.382 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fe7e6c3a640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fe7e6c3a640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:44.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fe7e6c3a640 10 monclient(hunting): picked mon.noname-a con 0x7fe7e0157780 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fe7e6c3a640 10 monclient(hunting): picked mon.noname-c con 0x7fe7e0157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fe7e6c3a640 10 monclient(hunting): picked mon.noname-b con 0x7fe7e0164d00 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fe7e6c3a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e005de70 con 0x7fe7e0157780 2026-02-25T15:01:44.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fe7e6c3a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e0055b20 con 0x7fe7e0164d00 2026-02-25T15:01:44.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fe7e6c3a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e0055e00 con 0x7fe7e0157b50 2026-02-25T15:01:44.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fe7e6c3a640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fe7e6c3a640 10 monclient(hunting): authenticate will time out at 2993.978136s 2026-02-25T15:01:44.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fa4b3c16640 1 Processor -- start 2026-02-25T15:01:44.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fa4b3c16640 1 -- start start 2026-02-25T15:01:44.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fa4b3c16640 10 monclient: init 2026-02-25T15:01:44.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fe7d7fff640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fe7e0157b50 0x7fe7e01645f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45758/0 (socket says 10.20.193.90:45758) 2026-02-25T15:01:44.385 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fe7d7fff640 1 -- 10.20.193.90:0/995246802 learned_addr learned my addr 10.20.193.90:0/995246802 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.386 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fa4b3c16640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.386 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fa4b3c16640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:44.386 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fa4b3c16640 10 monclient(hunting): picked mon.noname-c con 0x7fa4ac1598e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.386 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fa4b3c16640 10 monclient(hunting): picked mon.noname-b con 0x7fa4ac0b27f0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.386 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fa4b3c16640 10 monclient(hunting): picked mon.noname-a con 0x7fa4ac05a010 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.387 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fa4b3c16640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4ac05d230 con 0x7fa4ac05a010 2026-02-25T15:01:44.387 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fa4b3c16640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4ac0b1750 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.387 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fa4b3c16640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4ac0b1a30 con 0x7fa4ac1598e0 2026-02-25T15:01:44.387 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fa4b3c16640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.387 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.378+0000 7fa4b3c16640 10 monclient(hunting): authenticate will time out at 2993.978803s 2026-02-25T15:01:44.387 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.379+0000 7fa4b198b640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fa4ac1598e0 0x7fa4ac0575f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45770/0 (socket says 10.20.193.90:45770) 2026-02-25T15:01:44.387 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.379+0000 7fa4b198b640 1 -- 10.20.193.90:0/1445266174 learned_addr learned my addr 10.20.193.90:0/1445266174 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.383+0000 7fb86aee1640 10 monclient: get_monmap_and_config 2026-02-25T15:01:44.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.383+0000 7fb86aee1640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.388 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.383+0000 7fb86aee1640 10 monclient: monmap: 2026-02-25T15:01:44.388 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.389 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.389 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:44.383716+0000 2026-02-25T15:01:44.389 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:44.383716+0000 2026-02-25T15:01:44.389 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.389 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.389 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.389 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.389 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.389 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.389 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.383+0000 7fb86aee1640 1 Processor -- start 2026-02-25T15:01:44.389 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.383+0000 7fb86aee1640 1 -- start start 2026-02-25T15:01:44.389 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.383+0000 7fb86aee1640 10 monclient: init 2026-02-25T15:01:44.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7fb86aee1640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7fb86aee1640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:44.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7fb86aee1640 10 monclient(hunting): picked mon.noname-b con 0x7fb864159a70 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7fb86aee1640 10 monclient(hunting): picked mon.noname-c con 0x7fb864159e40 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7f062524e640 10 monclient: get_monmap_and_config 2026-02-25T15:01:44.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7f062524e640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7fb86aee1640 10 monclient(hunting): picked mon.noname-a con 0x7fb864058070 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7f062524e640 10 monclient: monmap: 2026-02-25T15:01:44.391 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.391 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.391 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:44.384725+0000 2026-02-25T15:01:44.391 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:44.384725+0000 2026-02-25T15:01:44.392 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.392 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.392 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.392 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.392 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.392 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7fb86aee1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb864163930 con 0x7fb864058070 2026-02-25T15:01:44.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7fb86aee1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb864163bf0 con 0x7fb864159a70 2026-02-25T15:01:44.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7fb86aee1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb864163ed0 con 0x7fb864159e40 2026-02-25T15:01:44.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7fb86aee1640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7fb86aee1640 10 monclient(hunting): authenticate will time out at 2993.984144s 2026-02-25T15:01:44.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7fb85bfff640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fb864159e40 0x7fb864163220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45780/0 (socket says 10.20.193.90:45780) 2026-02-25T15:01:44.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7fb85bfff640 1 -- 10.20.193.90:0/3455835525 learned_addr learned my addr 10.20.193.90:0/3455835525 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7f062524e640 1 Processor -- start 2026-02-25T15:01:44.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7f062524e640 1 -- start start 2026-02-25T15:01:44.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.384+0000 7f062524e640 10 monclient: init 2026-02-25T15:01:44.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.385+0000 7f062524e640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.385+0000 7f062524e640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:44.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.385+0000 7f062524e640 10 monclient(hunting): picked mon.noname-a con 0x7f0620159a70 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.385+0000 7f062524e640 10 monclient(hunting): picked mon.noname-b con 0x7f0620159e40 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.385+0000 7f062524e640 10 monclient(hunting): picked mon.noname-c con 0x7f0620058070 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.385+0000 7f062524e640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0620163930 con 0x7f0620159a70 2026-02-25T15:01:44.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.385+0000 7f062524e640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0620163bf0 con 0x7f0620159e40 2026-02-25T15:01:44.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.385+0000 7f062524e640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0620163ed0 con 0x7f0620058070 2026-02-25T15:01:44.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.385+0000 7f062524e640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.385+0000 7f062524e640 10 monclient(hunting): authenticate will time out at 2993.985405s 2026-02-25T15:01:44.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.385+0000 7f061f577640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f0620058070 0x7f0620058460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45786/0 (socket says 10.20.193.90:45786) 2026-02-25T15:01:44.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.385+0000 7f061f577640 1 -- 10.20.193.90:0/3815389404 learned_addr learned my addr 10.20.193.90:0/3815389404 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.392+0000 7fd58ccaa640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-25T15:01:44.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.392+0000 7fd58ccaa640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-25T15:01:44.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.392+0000 7fd58ccaa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.392+0000 7fd58ccaa640 1 -- 10.20.193.90:0/2898396181 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd58804b790 con 0x7fd5880ba350 2026-02-25T15:01:44.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.394+0000 7fd56effd640 1 -- 10.20.193.90:0/2898396181 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (unknown 1482059429 0 213165853) 0x7fd58804b790 con 0x7fd5880ba350 2026-02-25T15:01:44.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.394+0000 7fd56effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-25T15:01:44.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.394+0000 7fd56effd640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.397 INFO:teuthology.orchestra.run.trial090.stdout:94489280517 2026-02-25T15:01:44.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.396+0000 7fd58ccaa640 10 librados: watch_flush enter 2026-02-25T15:01:44.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.396+0000 7fd58ccaa640 10 librados: watch_flush exit 2026-02-25T15:01:44.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.396+0000 7fd58ccaa640 20 client.4598.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.396+0000 7fd58ccaa640 10 client.4598.objecter successfully canceled tick 2026-02-25T15:01:44.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.396+0000 7fd58ccaa640 1 -- 10.20.193.90:0/2898396181 >> v1:10.20.193.189:6800/3647375982 conn(0x7fd554079ea0 legacy=0x7fd55409a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.396+0000 7fd58ccaa640 10 monclient: shutdown 2026-02-25T15:01:44.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.396+0000 7fd58ccaa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.396+0000 7fd58ccaa640 1 -- 10.20.193.90:0/2898396181 >> v1:10.20.193.94:6789/0 conn(0x7fd5880ba350 legacy=0x7fd588161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.396+0000 7fd58ccaa640 1 -- 10.20.193.90:0/2898396181 shutdown_connections 2026-02-25T15:01:44.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.396+0000 7fd58ccaa640 1 -- 10.20.193.90:0/2898396181 >> 10.20.193.90:0/2898396181 conn(0x7fd5880b7990 msgr2=0x7fd5880b66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.396+0000 7fd58ccaa640 1 -- 10.20.193.90:0/2898396181 shutdown_connections 2026-02-25T15:01:44.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.396+0000 7fd58ccaa640 1 -- 10.20.193.90:0/2898396181 wait complete. 2026-02-25T15:01:44.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.396+0000 7fd58ccaa640 1 librados: shutdown 2026-02-25T15:01:44.407 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.7 2026-02-25T15:01:44.407 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:44.413 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2026-02-25T15:01:44.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.424+0000 7ff726ffd640 1 -- 10.20.193.90:0/54761343 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7ff73005f540 con 0x7ff740056090 2026-02-25T15:01:44.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.424+0000 7ff726ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.424+0000 7ff726ffd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:44.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.436+0000 7fab40eb7640 1 -- 10.20.193.90:0/4177226885 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2022936216 0 0) 0x7fab2c002910 con 0x7fab3c0b9f80 2026-02-25T15:01:44.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.436+0000 7fab40eb7640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.436+0000 7fab40eb7640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.436+0000 7fab40eb7640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.436+0000 7fab40eb7640 10 monclient(hunting): my global_id is 4590 2026-02-25T15:01:44.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.436+0000 7fab40eb7640 1 -- 10.20.193.90:0/4177226885 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab24003610 con 0x7fab3c0b9f80 2026-02-25T15:01:44.470 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.469+0000 7fab40eb7640 1 -- 10.20.193.90:0/4177226885 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3773068092 0 0) 0x7fab3c163bb0 con 0x7fab3c0585e0 2026-02-25T15:01:44.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.469+0000 7fab40eb7640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.469+0000 7fab40eb7640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.469+0000 7fab40eb7640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.469+0000 7fab40eb7640 10 monclient(hunting): my global_id is 4573 2026-02-25T15:01:44.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.469+0000 7fab40eb7640 1 -- 10.20.193.90:0/4177226885 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab2c002910 con 0x7fab3c0585e0 2026-02-25T15:01:44.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.469+0000 7fe7d77fe640 1 -- 10.20.193.90:0/995246802 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1675960143 0 0) 0x7fe7e0055b20 con 0x7fe7e0164d00 2026-02-25T15:01:44.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.469+0000 7fe7d77fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.469+0000 7fe7d77fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.469+0000 7fe7d77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.469+0000 7fe7d77fe640 10 monclient(hunting): my global_id is 4576 2026-02-25T15:01:44.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.469+0000 7fe7d77fe640 1 -- 10.20.193.90:0/995246802 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7cc003610 con 0x7fe7e0164d00 2026-02-25T15:01:44.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fe7d77fe640 1 -- 10.20.193.90:0/995246802 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4021188709 0 0) 0x7fe7e0055e00 con 0x7fe7e0157b50 2026-02-25T15:01:44.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fe7d77fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fe7d77fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fe7d77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fe7d77fe640 10 monclient(hunting): my global_id is 4607 2026-02-25T15:01:44.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fe7d77fe640 1 -- 10.20.193.90:0/995246802 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7e0055b20 con 0x7fe7e0157b50 2026-02-25T15:01:44.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fa4b0989640 1 -- 10.20.193.90:0/1445266174 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1363047210 0 0) 0x7fa4ac0b1750 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fa4b0989640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fa4b0989640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fa4b0989640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fa4b0989640 10 monclient(hunting): my global_id is 4579 2026-02-25T15:01:44.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fa4b0989640 1 -- 10.20.193.90:0/1445266174 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa494003610 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fa4b0989640 1 -- 10.20.193.90:0/1445266174 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 336867519 0 0) 0x7fa4ac0b1a30 con 0x7fa4ac1598e0 2026-02-25T15:01:44.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fa4b0989640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fa4b0989640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fa4b0989640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fa4b0989640 10 monclient(hunting): my global_id is 4610 2026-02-25T15:01:44.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7fa4b0989640 1 -- 10.20.193.90:0/1445266174 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4ac0b1750 con 0x7fa4ac1598e0 2026-02-25T15:01:44.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7f061dd74640 1 -- 10.20.193.90:0/3815389404 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 798063848 0 0) 0x7f0620163ed0 con 0x7f0620058070 2026-02-25T15:01:44.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7f061dd74640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7f061dd74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7f061dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7f061dd74640 10 monclient(hunting): my global_id is 4616 2026-02-25T15:01:44.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.470+0000 7f061dd74640 1 -- 10.20.193.90:0/3815389404 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f060c003610 con 0x7f0620058070 2026-02-25T15:01:44.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.471+0000 7fab40eb7640 1 -- 10.20.193.90:0/4177226885 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1433426652 0 0) 0x7fab3c163e90 con 0x7fab3c0ba350 2026-02-25T15:01:44.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.471+0000 7fab40eb7640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.471+0000 7fab40eb7640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.471+0000 7fab40eb7640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.471+0000 7fab40eb7640 10 monclient(hunting): my global_id is 4604 2026-02-25T15:01:44.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.471+0000 7fab40eb7640 1 -- 10.20.193.90:0/4177226885 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab3c163bb0 con 0x7fab3c0ba350 2026-02-25T15:01:44.478 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2026-02-25T15:01:44.478 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.471+0000 7fb85b7fe640 1 -- 10.20.193.90:0/3455835525 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3365229667 0 0) 0x7fb864163ed0 con 0x7fb864159e40 2026-02-25T15:01:44.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.471+0000 7fb85b7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.471+0000 7fb85b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.471+0000 7fb85b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.471+0000 7fb85b7fe640 10 monclient(hunting): my global_id is 4613 2026-02-25T15:01:44.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.471+0000 7fb85b7fe640 1 -- 10.20.193.90:0/3455835525 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb850003610 con 0x7fb864159e40 2026-02-25T15:01:44.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.474+0000 7fb85b7fe640 1 -- 10.20.193.90:0/3455835525 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1283353728 0 0) 0x7fb864163bf0 con 0x7fb864159a70 2026-02-25T15:01:44.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.474+0000 7fb85b7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.474+0000 7fb85b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.474+0000 7fb85b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.474+0000 7fb85b7fe640 10 monclient(hunting): my global_id is 4582 2026-02-25T15:01:44.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.474+0000 7fb85b7fe640 1 -- 10.20.193.90:0/3455835525 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb864163ed0 con 0x7fb864159a70 2026-02-25T15:01:44.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.474+0000 7f061dd74640 1 -- 10.20.193.90:0/3815389404 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3820845501 0 0) 0x7f0620163bf0 con 0x7f0620159e40 2026-02-25T15:01:44.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.474+0000 7f061dd74640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.474+0000 7f061dd74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.474+0000 7f061dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.474+0000 7f061dd74640 10 monclient(hunting): my global_id is 4585 2026-02-25T15:01:44.480 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.474+0000 7f061dd74640 1 -- 10.20.193.90:0/3815389404 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0620163ed0 con 0x7f0620159e40 2026-02-25T15:01:44.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab40eb7640 1 -- 10.20.193.90:0/4177226885 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 312616557 0 0) 0x7fab2c002910 con 0x7fab3c0585e0 2026-02-25T15:01:44.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab40eb7640 1 -- 10.20.193.90:0/4177226885 >> v1:10.20.193.94:6789/0 conn(0x7fab3c0ba350 legacy=0x7fab3c057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab40eb7640 1 -- 10.20.193.90:0/4177226885 >> v1:10.20.193.90:6789/0 conn(0x7fab3c0b9f80 legacy=0x7fab3c163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab40eb7640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab40eb7640 1 monclient: found mon.noname-b 2026-02-25T15:01:44.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab40eb7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab40eb7640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab40eb7640 1 -- 10.20.193.90:0/4177226885 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3c0573f0 con 0x7fab3c0585e0 2026-02-25T15:01:44.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab40eb7640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab40eb7640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.483 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab40eb7640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.483 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab40eb7640 1 -- 10.20.193.90:0/4177226885 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab38003170 con 0x7fab3c0585e0 2026-02-25T15:01:44.483 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab40eb7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.483 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab40eb7640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:44.483 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab40eb7640 10 monclient: dump: 2026-02-25T15:01:44.483 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.483 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.483 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.483 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.483 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.483 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.483 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.483 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.484 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.484 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.484 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab44144640 5 monclient: authenticate success, global_id 4573 2026-02-25T15:01:44.484 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fab44144640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.485 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b0989640 1 -- 10.20.193.90:0/1445266174 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 366921203 0 0) 0x7fa494003610 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.485 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b0989640 1 -- 10.20.193.90:0/1445266174 >> v1:10.20.193.94:6789/0 conn(0x7fa4ac1598e0 legacy=0x7fa4ac0575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.485 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b0989640 1 -- 10.20.193.90:0/1445266174 >> v1:10.20.193.90:6789/0 conn(0x7fa4ac05a010 legacy=0x7fa4ac1c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.485 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b0989640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.485 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b0989640 1 monclient: found mon.noname-b 2026-02-25T15:01:44.486 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b0989640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.486 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b0989640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.486 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b0989640 1 -- 10.20.193.90:0/1445266174 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ac0b1d10 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.486 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b0989640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.486 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b0989640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.486 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b0989640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.486 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b3c16640 5 monclient: authenticate success, global_id 4579 2026-02-25T15:01:44.486 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b3c16640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.487 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7d77fe640 1 -- 10.20.193.90:0/995246802 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3517566430 0 0) 0x7fe7cc003610 con 0x7fe7e0164d00 2026-02-25T15:01:44.488 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7d77fe640 1 -- 10.20.193.90:0/995246802 >> v1:10.20.193.94:6789/0 conn(0x7fe7e0157b50 legacy=0x7fe7e01645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.488 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7d77fe640 1 -- 10.20.193.90:0/995246802 >> v1:10.20.193.90:6789/0 conn(0x7fe7e0157780 legacy=0x7fe7e0056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.488 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7d77fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.488 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7d77fe640 1 monclient: found mon.noname-b 2026-02-25T15:01:44.488 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.488 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7d77fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.488 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7d77fe640 1 -- 10.20.193.90:0/995246802 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e00560e0 con 0x7fe7e0164d00 2026-02-25T15:01:44.488 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7d77fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.488 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7d77fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.488 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.488 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7d77fe640 1 -- 10.20.193.90:0/995246802 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe7dc003170 con 0x7fe7e0164d00 2026-02-25T15:01:44.489 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7d77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.489 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7d77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:44.489 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7d77fe640 10 monclient: dump: 2026-02-25T15:01:44.489 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.489 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.489 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.489 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.489 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.489 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.489 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.489 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.489 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.490 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.490 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7e6c3a640 5 monclient: authenticate success, global_id 4576 2026-02-25T15:01:44.490 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fe7e6c3a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.490 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b0989640 1 -- 10.20.193.90:0/1445266174 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa4a0002df0 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.491 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b0989640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.491 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b0989640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:44.491 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b0989640 10 monclient: dump: 2026-02-25T15:01:44.491 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.491 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.491 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.491 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.491 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.491 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.491 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.491 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.491 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.492 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.492 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.475+0000 7fa4b3c16640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.492 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.476+0000 7fb85b7fe640 1 -- 10.20.193.90:0/3455835525 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 888543744 0 0) 0x7fb864163930 con 0x7fb864058070 2026-02-25T15:01:44.492 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.476+0000 7fb85b7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.493 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.476+0000 7fb85b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.493 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.476+0000 7fb85b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.493 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.476+0000 7fb85b7fe640 10 monclient(hunting): my global_id is 4596 2026-02-25T15:01:44.493 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.476+0000 7fb85b7fe640 1 -- 10.20.193.90:0/3455835525 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb864163bf0 con 0x7fb864058070 2026-02-25T15:01:44.494 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.478+0000 7f061dd74640 1 -- 10.20.193.90:0/3815389404 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3032528743 0 0) 0x7f0620163930 con 0x7f0620159a70 2026-02-25T15:01:44.494 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.478+0000 7f061dd74640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.494 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.478+0000 7f061dd74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.494 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.478+0000 7f061dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.494 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.478+0000 7f061dd74640 10 monclient(hunting): my global_id is 4599 2026-02-25T15:01:44.494 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.478+0000 7f061dd74640 1 -- 10.20.193.90:0/3815389404 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0620163bf0 con 0x7f0620159a70 2026-02-25T15:01:44.495 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb85b7fe640 1 -- 10.20.193.90:0/3455835525 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 617197647 0 0) 0x7fb850003610 con 0x7fb864159e40 2026-02-25T15:01:44.495 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb85b7fe640 1 -- 10.20.193.90:0/3455835525 >> v1:10.20.193.93:6789/0 conn(0x7fb864159a70 legacy=0x7fb8640b6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.495 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb85b7fe640 1 -- 10.20.193.90:0/3455835525 >> v1:10.20.193.90:6789/0 conn(0x7fb864058070 legacy=0x7fb864058460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.496 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb85b7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.496 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb85b7fe640 1 monclient: found mon.noname-c 2026-02-25T15:01:44.496 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb85b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.496 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb85b7fe640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.496 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb85b7fe640 1 -- 10.20.193.90:0/3455835525 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb864057590 con 0x7fb864159e40 2026-02-25T15:01:44.496 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb85b7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.496 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb85b7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.496 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb85b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.496 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb85b7fe640 1 -- 10.20.193.90:0/3455835525 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb84c002df0 con 0x7fb864159e40 2026-02-25T15:01:44.496 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb85b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.497 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb85b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:44.497 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb85b7fe640 10 monclient: dump: 2026-02-25T15:01:44.497 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.497 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.497 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.497 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.497 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.497 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.497 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.497 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.497 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.497 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.498 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb86aee1640 5 monclient: authenticate success, global_id 4613 2026-02-25T15:01:44.498 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.480+0000 7fb86aee1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.498 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-25T15:01:44.499 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f061dd74640 1 -- 10.20.193.90:0/3815389404 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2331080535 0 0) 0x7f0620163ed0 con 0x7f0620159e40 2026-02-25T15:01:44.499 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f061dd74640 1 -- 10.20.193.90:0/3815389404 >> v1:10.20.193.94:6789/0 conn(0x7f0620058070 legacy=0x7f0620058460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.499 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f061dd74640 1 -- 10.20.193.90:0/3815389404 >> v1:10.20.193.90:6789/0 conn(0x7f0620159a70 legacy=0x7f06200b6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.500 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f061dd74640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.500 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f061dd74640 1 monclient: found mon.noname-b 2026-02-25T15:01:44.500 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f061dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.500 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f061dd74640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.500 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f061dd74640 1 -- 10.20.193.90:0/3815389404 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0620057590 con 0x7f0620159e40 2026-02-25T15:01:44.500 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f061dd74640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.500 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f061dd74640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.500 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f061dd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.500 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f061dd74640 1 -- 10.20.193.90:0/3815389404 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0608002df0 con 0x7f0620159e40 2026-02-25T15:01:44.500 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f061dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.500 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f061dd74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:44.500 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f061dd74640 10 monclient: dump: 2026-02-25T15:01:44.500 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.500 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.501 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.501 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.501 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.501 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.501 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.501 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.501 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.501 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.501 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f062524e640 5 monclient: authenticate success, global_id 4585 2026-02-25T15:01:44.501 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7f062524e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.502 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7fab40eb7640 1 -- 10.20.193.90:0/4177226885 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fab38003340 con 0x7fab3c0585e0 2026-02-25T15:01:44.502 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7fab40eb7640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.502 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7fab40eb7640 1 -- 10.20.193.90:0/4177226885 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab38003780 con 0x7fab3c0585e0 2026-02-25T15:01:44.502 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7fab40eb7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.502 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7fab40eb7640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:44.502 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7fab40eb7640 10 monclient: dump: 2026-02-25T15:01:44.502 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.502 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.502 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.502 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.503 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.503 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.503 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.503 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.503 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.503 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7fab44144640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:44.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.481+0000 7fab44144640 4 set_mon_vals no callback set 2026-02-25T15:01:44.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fe7d77fe640 1 -- 10.20.193.90:0/995246802 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe7dc003380 con 0x7fe7e0164d00 2026-02-25T15:01:44.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fe7d77fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fe7d77fe640 1 -- 10.20.193.90:0/995246802 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe7dc0037c0 con 0x7fe7e0164d00 2026-02-25T15:01:44.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fe7d77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fe7d77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:44.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fe7d77fe640 10 monclient: dump: 2026-02-25T15:01:44.504 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.504 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.504 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.504 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.504 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.504 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.504 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.505 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.505 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.505 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.505 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fe7e6c3a640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:44.505 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fe7e6c3a640 4 set_mon_vals no callback set 2026-02-25T15:01:44.505 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fab44144640 10 monclient: shutdown 2026-02-25T15:01:44.506 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fab44144640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.506 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fab44144640 1 -- 10.20.193.90:0/4177226885 >> v1:10.20.193.93:6789/0 conn(0x7fab3c0585e0 legacy=0x7fab3c166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.506 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fab44144640 1 -- 10.20.193.90:0/4177226885 shutdown_connections 2026-02-25T15:01:44.506 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fab44144640 1 -- 10.20.193.90:0/4177226885 >> 10.20.193.90:0/4177226885 conn(0x7fab3c0b7990 msgr2=0x7fab3c0b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.506 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fab44144640 1 -- 10.20.193.90:0/4177226885 shutdown_connections 2026-02-25T15:01:44.506 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fab44144640 1 -- 10.20.193.90:0/4177226885 wait complete. 2026-02-25T15:01:44.507 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fe7e6c3a640 10 monclient: shutdown 2026-02-25T15:01:44.507 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fe7e6c3a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.507 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fe7e6c3a640 1 -- 10.20.193.90:0/995246802 >> v1:10.20.193.93:6789/0 conn(0x7fe7e0164d00 legacy=0x7fe7e01c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.507 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fe7e6c3a640 1 -- 10.20.193.90:0/995246802 shutdown_connections 2026-02-25T15:01:44.507 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fe7e6c3a640 1 -- 10.20.193.90:0/995246802 >> 10.20.193.90:0/995246802 conn(0x7fe7e00b2ef0 msgr2=0x7fe7e00b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.507 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fe7e6c3a640 1 -- 10.20.193.90:0/995246802 shutdown_connections 2026-02-25T15:01:44.507 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fe7e6c3a640 1 -- 10.20.193.90:0/995246802 wait complete. 2026-02-25T15:01:44.508 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fa4b0989640 1 -- 10.20.193.90:0/1445266174 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa4a0003040 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.508 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fa4b0989640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.508 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fa4b0989640 1 -- 10.20.193.90:0/1445266174 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa4a0003480 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.508 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fa4b0989640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.508 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fa4b0989640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:44.508 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fa4b0989640 10 monclient: dump: 2026-02-25T15:01:44.508 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.508 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.508 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.508 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.508 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.508 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.509 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.509 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.509 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.509 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.509 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fa4b3c16640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:44.509 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fa4b3c16640 4 set_mon_vals no callback set 2026-02-25T15:01:44.509 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fab44144640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.482+0000 7fab44144640 10 monclient: monmap: 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 1 librados: starting msgr at 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 1 librados: starting objecter 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 1 Processor -- start 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 1 -- start start 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 1 librados: setting wanted keys 2026-02-25T15:01:44.510 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 1 librados: calling monclient init 2026-02-25T15:01:44.511 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 10 monclient: init 2026-02-25T15:01:44.511 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fa4b3c16640 10 monclient: shutdown 2026-02-25T15:01:44.511 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fa4b3c16640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.511 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fa4b3c16640 1 -- 10.20.193.90:0/1445266174 >> v1:10.20.193.93:6789/0 conn(0x7fa4ac0b27f0 legacy=0x7fa4ac059900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.511 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fa4b3c16640 1 -- 10.20.193.90:0/1445266174 shutdown_connections 2026-02-25T15:01:44.511 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fa4b3c16640 1 -- 10.20.193.90:0/1445266174 >> 10.20.193.90:0/1445266174 conn(0x7fa4ac0b7120 msgr2=0x7fa4ac0b7520 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.511 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fe7e6c3a640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.512 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fe7e6c3a640 10 monclient: monmap: 2026-02-25T15:01:44.512 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.512 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:44.512 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:44.512 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:44.512 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.512 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:44.512 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.512 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.512 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.512 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.512 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fa4b3c16640 1 -- 10.20.193.90:0/1445266174 shutdown_connections 2026-02-25T15:01:44.513 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fa4b3c16640 1 -- 10.20.193.90:0/1445266174 wait complete. 2026-02-25T15:01:44.513 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fe7e6c3a640 1 librados: starting msgr at 2026-02-25T15:01:44.513 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fe7e6c3a640 1 librados: starting objecter 2026-02-25T15:01:44.513 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fe7e6c3a640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:44.513 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fe7e6c3a640 1 Processor -- start 2026-02-25T15:01:44.513 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fb85b7fe640 1 -- 10.20.193.90:0/3455835525 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb84c002fc0 con 0x7fb864159e40 2026-02-25T15:01:44.513 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fb85b7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fb85b7fe640 1 -- 10.20.193.90:0/3455835525 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb84c003400 con 0x7fb864159e40 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fb85b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fb85b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fb85b7fe640 10 monclient: dump: 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fb86aee1640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:44.514 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fb86aee1640 4 set_mon_vals no callback set 2026-02-25T15:01:44.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fe7e6c3a640 1 -- start start 2026-02-25T15:01:44.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fe7e6c3a640 1 librados: setting wanted keys 2026-02-25T15:01:44.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fe7e6c3a640 1 librados: calling monclient init 2026-02-25T15:01:44.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fe7e6c3a640 10 monclient: init 2026-02-25T15:01:44.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:44.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 10 monclient(hunting): picked mon.noname-a con 0x7fab3c0585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 10 monclient(hunting): picked mon.noname-b con 0x7fab3c0b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 10 monclient(hunting): picked mon.noname-c con 0x7fab3c0ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c1fef50 con 0x7fab3c0585e0 2026-02-25T15:01:44.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c1605d0 con 0x7fab3c0b9f80 2026-02-25T15:01:44.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c1608b0 con 0x7fab3c0ba350 2026-02-25T15:01:44.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fab44144640 10 monclient(hunting): authenticate will time out at 2994.083827s 2026-02-25T15:01:44.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fa4b3c16640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.483+0000 7fa4b3c16640 10 monclient: monmap: 2026-02-25T15:01:44.516 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.517 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:44.517 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:44.517 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:44.517 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.517 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:44.517 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.517 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.517 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.517 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fe7e6c3a640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 1 librados: starting msgr at 2026-02-25T15:01:44.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 1 librados: starting objecter 2026-02-25T15:01:44.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fe7e6c3a640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:44.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fe7e6c3a640 10 monclient(hunting): picked mon.noname-c con 0x7fe7e0157780 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fe7e6c3a640 10 monclient(hunting): picked mon.noname-a con 0x7fe7e0157b50 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fe7e6c3a640 10 monclient(hunting): picked mon.noname-b con 0x7fe7e0164d00 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fe7e6c3a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e012a090 con 0x7fe7e0157b50 2026-02-25T15:01:44.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fe7e6c3a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e011f280 con 0x7fe7e0164d00 2026-02-25T15:01:44.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fe7e6c3a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e011f560 con 0x7fe7e0157780 2026-02-25T15:01:44.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fe7e6c3a640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 10 monclient: shutdown 2026-02-25T15:01:44.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fe7e6c3a640 10 monclient(hunting): authenticate will time out at 2994.084132s 2026-02-25T15:01:44.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:44.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 1 -- 10.20.193.90:0/3455835525 >> v1:10.20.193.94:6789/0 conn(0x7fb864159e40 legacy=0x7fb864163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 1 Processor -- start 2026-02-25T15:01:44.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 1 -- 10.20.193.90:0/3455835525 shutdown_connections 2026-02-25T15:01:44.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 1 -- 10.20.193.90:0/3455835525 >> 10.20.193.90:0/3455835525 conn(0x7fb8640b7780 msgr2=0x7fb8640b7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 1 -- 10.20.193.90:0/3455835525 shutdown_connections 2026-02-25T15:01:44.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 1 -- start start 2026-02-25T15:01:44.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 1 librados: setting wanted keys 2026-02-25T15:01:44.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 1 -- 10.20.193.90:0/3455835525 wait complete. 2026-02-25T15:01:44.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 1 librados: calling monclient init 2026-02-25T15:01:44.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 10 monclient: init 2026-02-25T15:01:44.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fab426ba640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fab3c0ba350 0x7fab3c1619c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45790/0 (socket says 10.20.193.90:45790) 2026-02-25T15:01:44.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fab426ba640 1 -- 10.20.193.90:0/2218227676 learned_addr learned my addr 10.20.193.90:0/2218227676 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 10 monclient: monmap: 2026-02-25T15:01:44.521 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.521 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:44.521 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:44.521 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:44.521 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.521 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:44.521 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.521 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.521 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.521 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fe7d7fff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fe7e0157b50 0x7fe7e01226b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57382/0 (socket says 10.20.193.90:57382) 2026-02-25T15:01:44.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fe7d7fff640 1 -- 10.20.193.90:0/2051178685 learned_addr learned my addr 10.20.193.90:0/2051178685 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fab2a7fc640 10 client.?.objecter ms_handle_connect 0x7fab3c0ba350 2026-02-25T15:01:44.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:44.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fab2a7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 10 monclient(hunting): picked mon.noname-c con 0x7fa4ac05a010 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 10 monclient(hunting): picked mon.noname-b con 0x7fa4ac0b27f0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 10 monclient(hunting): picked mon.noname-a con 0x7fa4ac1598e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4ac0b5d40 con 0x7fa4ac1598e0 2026-02-25T15:01:44.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4ac0b6020 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4ac0b6300 con 0x7fa4ac05a010 2026-02-25T15:01:44.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fa4b3c16640 10 monclient(hunting): authenticate will time out at 2994.084840s 2026-02-25T15:01:44.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 1 librados: starting msgr at 2026-02-25T15:01:44.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 1 librados: starting objecter 2026-02-25T15:01:44.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:44.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 1 Processor -- start 2026-02-25T15:01:44.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 1 -- start start 2026-02-25T15:01:44.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 1 librados: setting wanted keys 2026-02-25T15:01:44.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 1 librados: calling monclient init 2026-02-25T15:01:44.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fb86aee1640 10 monclient: init 2026-02-25T15:01:44.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fab2a7fc640 10 client.?.objecter ms_handle_connect 0x7fab3c0585e0 2026-02-25T15:01:44.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.484+0000 7fab2a7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7fe7d57fa640 10 client.?.objecter ms_handle_connect 0x7fe7e0157b50 2026-02-25T15:01:44.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7fe7d57fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7fb86aee1640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7fb86aee1640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:44.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7fb86aee1640 10 monclient(hunting): picked mon.noname-a con 0x7fb864058070 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7fb86aee1640 10 monclient(hunting): picked mon.noname-c con 0x7fb864159a70 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7fb86aee1640 10 monclient(hunting): picked mon.noname-b con 0x7fb864159e40 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7fb86aee1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8641606c0 con 0x7fb864058070 2026-02-25T15:01:44.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7fb86aee1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb86415e8d0 con 0x7fb864159e40 2026-02-25T15:01:44.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7fb86aee1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb86415ebb0 con 0x7fb864159a70 2026-02-25T15:01:44.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7fb86aee1640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7fb86aee1640 10 monclient(hunting): authenticate will time out at 2994.085419s 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7f061dd74640 1 -- 10.20.193.90:0/3815389404 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0608002f90 con 0x7f0620159e40 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7f061dd74640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7f061dd74640 1 -- 10.20.193.90:0/3815389404 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f06080033d0 con 0x7f0620159e40 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7f061dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7f061dd74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7f061dd74640 10 monclient: dump: 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7f062524e640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:44.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.485+0000 7f062524e640 4 set_mon_vals no callback set 2026-02-25T15:01:44.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 10 monclient: shutdown 2026-02-25T15:01:44.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 1 -- 10.20.193.90:0/3815389404 >> v1:10.20.193.93:6789/0 conn(0x7f0620159e40 legacy=0x7f0620163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 1 -- 10.20.193.90:0/3815389404 shutdown_connections 2026-02-25T15:01:44.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 1 -- 10.20.193.90:0/3815389404 >> 10.20.193.90:0/3815389404 conn(0x7f06200b7780 msgr2=0x7f06200b7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 1 -- 10.20.193.90:0/3815389404 shutdown_connections 2026-02-25T15:01:44.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 1 -- 10.20.193.90:0/3815389404 wait complete. 2026-02-25T15:01:44.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7fe7d57fa640 10 client.?.objecter ms_handle_connect 0x7fe7e0157780 2026-02-25T15:01:44.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7fe7d57fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7fb869457640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fb864159e40 0x7fb8641fc960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:51176/0 (socket says 10.20.193.90:51176) 2026-02-25T15:01:44.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7fb869457640 1 -- 10.20.193.90:0/67494453 learned_addr learned my addr 10.20.193.90:0/67494453 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7fab2a7fc640 10 client.?.objecter ms_handle_connect 0x7fab3c0b9f80 2026-02-25T15:01:44.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7fab2a7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7fa4b198b640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fa4ac05a010 0x7fa4ac1fb8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45812/0 (socket says 10.20.193.90:45812) 2026-02-25T15:01:44.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7fa4b198b640 1 -- 10.20.193.90:0/2662844873 learned_addr learned my addr 10.20.193.90:0/2662844873 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 10 monclient: monmap: 2026-02-25T15:01:44.528 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.528 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:44.528 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:44.528 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:44.528 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.528 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:44.528 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.528 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.528 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.528 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7fe7d57fa640 10 client.?.objecter ms_handle_connect 0x7fe7e0164d00 2026-02-25T15:01:44.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7fe7d57fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 1 librados: starting msgr at 2026-02-25T15:01:44.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 1 librados: starting objecter 2026-02-25T15:01:44.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:44.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 1 Processor -- start 2026-02-25T15:01:44.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7fb8597fa640 10 client.?.objecter ms_handle_connect 0x7fb864159e40 2026-02-25T15:01:44.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 1 -- start start 2026-02-25T15:01:44.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 1 librados: setting wanted keys 2026-02-25T15:01:44.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 1 librados: calling monclient init 2026-02-25T15:01:44.529 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7f062524e640 10 monclient: init 2026-02-25T15:01:44.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7fb8597fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7fa49a7fc640 10 client.?.objecter ms_handle_connect 0x7fa4ac05a010 2026-02-25T15:01:44.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.486+0000 7fa49a7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7f062524e640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7f062524e640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:44.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7f062524e640 10 monclient(hunting): picked mon.noname-b con 0x7f0620058070 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7f062524e640 10 monclient(hunting): picked mon.noname-c con 0x7f0620159a70 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7f062524e640 10 monclient(hunting): picked mon.noname-a con 0x7f0620159e40 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7f062524e640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06200b9820 con 0x7f0620159e40 2026-02-25T15:01:44.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7f062524e640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f062013a1e0 con 0x7f0620058070 2026-02-25T15:01:44.530 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7f062524e640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f062013a4c0 con 0x7f0620159a70 2026-02-25T15:01:44.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7f062524e640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7f062524e640 10 monclient(hunting): authenticate will time out at 2994.087310s 2026-02-25T15:01:44.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7fb8597fa640 10 client.?.objecter ms_handle_connect 0x7fb864058070 2026-02-25T15:01:44.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7fb8597fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 798540384 0 0) 0x7fe7e012a090 con 0x7fe7e0157b50 2026-02-25T15:01:44.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7fe7d57fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7fe7d57fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7fe7d57fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7fe7d57fa640 10 monclient(hunting): my global_id is 4608 2026-02-25T15:01:44.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7b0003610 con 0x7fe7e0157b50 2026-02-25T15:01:44.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7fa49a7fc640 10 client.?.objecter ms_handle_connect 0x7fa4ac1598e0 2026-02-25T15:01:44.531 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.487+0000 7fa49a7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 832296256 0 0) 0x7fab3c1608b0 con 0x7fab3c0ba350 2026-02-25T15:01:44.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fab2a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fab2a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fab2a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fab2a7fc640 10 monclient(hunting): my global_id is 4619 2026-02-25T15:01:44.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab0c003610 con 0x7fab3c0ba350 2026-02-25T15:01:44.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2420511194 0 0) 0x7fe7e011f560 con 0x7fe7e0157780 2026-02-25T15:01:44.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fe7d57fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fe7d57fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fe7d57fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fe7d57fa640 10 monclient(hunting): my global_id is 4622 2026-02-25T15:01:44.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7e012a090 con 0x7fe7e0157780 2026-02-25T15:01:44.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2880382418 0 0) 0x7fa4ac0b6300 con 0x7fa4ac05a010 2026-02-25T15:01:44.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fa49a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fa49a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fa49a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fa49a7fc640 10 monclient(hunting): my global_id is 4625 2026-02-25T15:01:44.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.488+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa47c003610 con 0x7fa4ac05a010 2026-02-25T15:01:44.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.489+0000 7f061e575640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f0620159a70 0x7f062015b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45836/0 (socket says 10.20.193.90:45836) 2026-02-25T15:01:44.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.489+0000 7f061e575640 1 -- 10.20.193.90:0/3632194069 learned_addr learned my addr 10.20.193.90:0/3632194069 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.489+0000 7fa49a7fc640 10 client.?.objecter ms_handle_connect 0x7fa4ac0b27f0 2026-02-25T15:01:44.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.489+0000 7fa49a7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.489+0000 7fb8597fa640 10 client.?.objecter ms_handle_connect 0x7fb864159a70 2026-02-25T15:01:44.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.489+0000 7fb8597fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.489+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737611637 0 0) 0x7fb86415e8d0 con 0x7fb864159e40 2026-02-25T15:01:44.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.489+0000 7fb8597fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.489+0000 7fb8597fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.489+0000 7fb8597fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.489+0000 7fb8597fa640 10 monclient(hunting): my global_id is 4591 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.489+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb834003610 con 0x7fb864159e40 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.490+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3057670340 0 0) 0x7fe7e011f280 con 0x7fe7e0164d00 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.490+0000 7fe7d57fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.490+0000 7fe7d57fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.490+0000 7fe7d57fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.490+0000 7fe7d57fa640 10 monclient(hunting): my global_id is 4594 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.490+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7e011f560 con 0x7fe7e0164d00 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.491+0000 7f05ff7fe640 10 client.?.objecter ms_handle_connect 0x7f0620159a70 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.491+0000 7f05ff7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.491+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2020657471 0 0) 0x7fab3c1fef50 con 0x7fab3c0585e0 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.491+0000 7fab2a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.491+0000 7fab2a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.491+0000 7fab2a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.491+0000 7fab2a7fc640 10 monclient(hunting): my global_id is 4605 2026-02-25T15:01:44.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.491+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab3c1608b0 con 0x7fab3c0585e0 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.491+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2239370207 0 0) 0x7fe7b0003610 con 0x7fe7e0157b50 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.491+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe7e011f280 con 0x7fe7e0157b50 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.491+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe7c80030f0 con 0x7fe7e0157b50 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.491+0000 7fe7d57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.491+0000 7fe7d57fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.491+0000 7fe7d57fa640 10 monclient(hunting): dump: 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.491+0000 7fe7d57fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.492+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2877764121 0 0) 0x7fb8641606c0 con 0x7fb864058070 2026-02-25T15:01:44.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.492+0000 7fb8597fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.492+0000 7fb8597fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.492+0000 7fb8597fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.492+0000 7fb8597fa640 10 monclient(hunting): my global_id is 4611 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.492+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb86415e8d0 con 0x7fb864058070 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.492+0000 7f7653ff5640 10 monclient: get_monmap_and_config 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.492+0000 7f7653ff5640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.492+0000 7f7653ff5640 10 monclient: monmap: 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:44.492779+0000 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:44.492779+0000 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.492+0000 7f7653ff5640 1 Processor -- start 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.492+0000 7f7653ff5640 1 -- start start 2026-02-25T15:01:44.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.492+0000 7f7653ff5640 10 monclient: init 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3335546197 0 0) 0x7fab0c003610 con 0x7fab3c0ba350 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fab3c1fef50 con 0x7fab3c0ba350 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab380037b0 con 0x7fab3c0ba350 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7fab2a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7fab2a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7fab2a7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7fab2a7fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7f7653ff5640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7f7653ff5640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7f7653ff5640 10 monclient(hunting): picked mon.noname-a con 0x7f764c0b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7f7653ff5640 10 monclient(hunting): picked mon.noname-c con 0x7f764c0ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7f7653ff5640 10 monclient(hunting): picked mon.noname-b con 0x7f764c0585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7f7653ff5640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f764c163930 con 0x7f764c0b9f80 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7f7653ff5640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f764c163bb0 con 0x7f764c0585e0 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7f7653ff5640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f764c163e90 con 0x7f764c0ba350 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7f7653ff5640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7f7653ff5640 10 monclient(hunting): authenticate will time out at 2994.093354s 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3240544583 0 0) 0x7fb86415ebb0 con 0x7fb864159a70 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7fb8597fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7fb8597fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7fb8597fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7fb8597fa640 10 monclient(hunting): my global_id is 4628 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.493+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8641606c0 con 0x7fb864159a70 2026-02-25T15:01:44.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.494+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2980707772 0 0) 0x7f062013a4c0 con 0x7f0620159a70 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.494+0000 7f05ff7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.494+0000 7f05ff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.494+0000 7f05ff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.494+0000 7f05ff7fe640 10 monclient(hunting): my global_id is 4631 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.494+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05ec003610 con 0x7f0620159a70 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.495+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3880089928 0 0) 0x7fa4ac0b5d40 con 0x7fa4ac1598e0 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.495+0000 7fa49a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.495+0000 7fa49a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.495+0000 7fa49a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.495+0000 7fa49a7fc640 10 monclient(hunting): my global_id is 4614 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.495+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4ac0b6300 con 0x7fa4ac1598e0 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.495+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3402584034 0 0) 0x7fb834003610 con 0x7fb864159e40 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.495+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb86415ebb0 con 0x7fb864159e40 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.495+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb8600033c0 con 0x7fb864159e40 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.495+0000 7fb8597fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.495+0000 7fb8597fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.495+0000 7fb8597fa640 10 monclient(hunting): dump: 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.539 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.495+0000 7fb8597fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.495+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1691457626 0 0) 0x7fe7e011f560 con 0x7fe7e0164d00 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe7b0003610 con 0x7fe7e0164d00 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe7dc0036d0 con 0x7fe7e0164d00 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fe7d57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fe7d57fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fe7d57fa640 10 monclient(hunting): dump: 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1459522685 0 0) 0x7fab3c1605d0 con 0x7fab3c0b9f80 2026-02-25T15:01:44.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fab2a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fab2a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fab2a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fab2a7fc640 10 monclient(hunting): my global_id is 4588 2026-02-25T15:01:44.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab0c003610 con 0x7fab3c0b9f80 2026-02-25T15:01:44.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1458943225 0 0) 0x7fa4ac0b6020 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fa49a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fa49a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fa49a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fa49a7fc640 10 monclient(hunting): my global_id is 4597 2026-02-25T15:01:44.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4ac0b5d40 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7f765256b640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f764c0585e0 0x7f764c166cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:51182/0 (socket says 10.20.193.90:51182) 2026-02-25T15:01:44.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7f765256b640 1 -- 10.20.193.90:0/3614549764 learned_addr learned my addr 10.20.193.90:0/3614549764 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7f05ff7fe640 10 client.?.objecter ms_handle_connect 0x7f0620058070 2026-02-25T15:01:44.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.496+0000 7f05ff7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.498+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3663204511 0 0) 0x7fb86415e8d0 con 0x7fb864058070 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.498+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb834003610 con 0x7fb864058070 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.498+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb854002ff0 con 0x7fb864058070 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.498+0000 7fb8597fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.498+0000 7fb8597fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.498+0000 7fb8597fa640 10 monclient(hunting): dump: 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.498+0000 7f05ff7fe640 10 client.?.objecter ms_handle_connect 0x7f0620159e40 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.498+0000 7f05ff7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.499+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1362964820 0 0) 0x7fb8641606c0 con 0x7fb864159a70 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.499+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb86415e8d0 con 0x7fb864159a70 2026-02-25T15:01:44.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.499+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb84c003170 con 0x7fb864159a70 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.499+0000 7fb8597fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.499+0000 7fb8597fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.499+0000 7fb8597fa640 10 monclient(hunting): dump: 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.500+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2083080780 0 0) 0x7f062013a1e0 con 0x7f0620058070 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.500+0000 7f05ff7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.500+0000 7f05ff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.500+0000 7f05ff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.500+0000 7f05ff7fe640 10 monclient(hunting): my global_id is 4600 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.500+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f062013a4c0 con 0x7f0620058070 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2550331848 0 0) 0x7fab3c1608b0 con 0x7fab3c0585e0 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fab3c1605d0 con 0x7fab3c0585e0 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab2c003280 con 0x7fab3c0585e0 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fab2a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fab2a7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fab2a7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.543 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1277682333 0 0) 0x7fe7e011f280 con 0x7fe7e0157b50 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 >> v1:10.20.193.94:6789/0 conn(0x7fe7e0157780 legacy=0x7fe7e0120a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 >> v1:10.20.193.93:6789/0 conn(0x7fe7e0164d00 legacy=0x7fe7e00aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7d57fa640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7d57fa640 1 monclient: found mon.a 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7d57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7d57fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e011f840 con 0x7fe7e0157b50 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7d57fa640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7d57fa640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7d57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7e6c3a640 5 monclient: authenticate success, global_id 4608 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7e6c3a640 10 monclient: _renew_subs 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7e6c3a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7e6c3a640 1 -- 10.20.193.90:0/2051178685 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e011fb20 con 0x7fe7e0157b50 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7e6c3a640 10 client.4608.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7e6c3a640 10 monclient: _renew_subs 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7e6c3a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7e6c3a640 1 -- 10.20.193.90:0/2051178685 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe7e0122fa0 con 0x7fe7e0157b50 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.502+0000 7fe7e6c3a640 1 librados: init done 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.503+0000 7fe7e6c3a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.503+0000 7fe7e6c3a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.503+0000 7fe7e6c3a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.503+0000 7fe7e6c3a640 1 -- 10.20.193.90:0/2051178685 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e00b0ac0 con 0x7fe7e0157b50 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.503+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1544028047 0 0) 0x7fab0c003610 con 0x7fab3c0b9f80 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.503+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fab3c1608b0 con 0x7fab3c0b9f80 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.503+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab30002f80 con 0x7fab3c0b9f80 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.503+0000 7fab2a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.503+0000 7fab2a7fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.503+0000 7fab2a7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.545 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3693887449 0 0) 0x7f06200b9820 con 0x7f0620159e40 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7f05ff7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7f05ff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7f05ff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7f05ff7fe640 10 monclient(hunting): my global_id is 4617 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f062013a1e0 con 0x7f0620159e40 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2144652519 0 0) 0x7fb86415ebb0 con 0x7fb864159e40 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 >> v1:10.20.193.94:6789/0 conn(0x7fb864159a70 legacy=0x7fb86415ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 >> v1:10.20.193.90:6789/0 conn(0x7fb864058070 legacy=0x7fb8641fbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb8597fa640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb8597fa640 1 monclient: found mon.b 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb8597fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb8597fa640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86415ee90 con 0x7fb864159e40 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb8597fa640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb8597fa640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb8597fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb86aee1640 5 monclient: authenticate success, global_id 4591 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb86aee1640 10 monclient: _renew_subs 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb86aee1640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb86aee1640 1 -- 10.20.193.90:0/67494453 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb86415f170 con 0x7fb864159e40 2026-02-25T15:01:44.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb86aee1640 10 client.4591.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb86aee1640 10 monclient: _renew_subs 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb86aee1640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb86aee1640 1 -- 10.20.193.90:0/67494453 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb8641fd070 con 0x7fb864159e40 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.504+0000 7fb86aee1640 1 librados: init done 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4154177355 0 0) 0x7fa47c003610 con 0x7fa4ac05a010 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa4ac0b6020 con 0x7fa4ac05a010 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa49c003050 con 0x7fa4ac05a010 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7fa49a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7fa49a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7fa49a7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7fa49a7fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:44.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 421777596 0 0) 0x7f05ec003610 con 0x7f0620159a70 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f06200b9820 con 0x7f0620159a70 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0608003360 con 0x7f0620159a70 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7f05ff7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7f05ff7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7f05ff7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7f05ff7fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7fb86aee1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7fb86aee1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7fb86aee1640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.505+0000 7fb86aee1640 1 -- 10.20.193.90:0/67494453 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8641fd300 con 0x7fb864159e40 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.506+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 842441428 0 0) 0x7fa4ac0b5d40 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.506+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa47c003610 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.506+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa4a0003390 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.506+0000 7fa49a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.506+0000 7fa49a7fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:44.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.506+0000 7fa49a7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.507+0000 7f7650d68640 1 -- 10.20.193.90:0/3614549764 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4130120373 0 0) 0x7f764c163bb0 con 0x7f764c0585e0 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.507+0000 7f7650d68640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.507+0000 7f7650d68640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.507+0000 7f7650d68640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.507+0000 7f7650d68640 10 monclient(hunting): my global_id is 4603 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.507+0000 7f7650d68640 1 -- 10.20.193.90:0/3614549764 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7634003610 con 0x7f764c0585e0 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 398894048 0 0) 0x7f06200b9820 con 0x7f0620159a70 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 >> v1:10.20.193.93:6789/0 conn(0x7f0620058070 legacy=0x7f06200b9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 >> v1:10.20.193.90:6789/0 conn(0x7f0620159e40 legacy=0x7f06201658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f05ff7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f05ff7fe640 1 monclient: found mon.c 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f05ff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f05ff7fe640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062013a7a0 con 0x7f0620159a70 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f05ff7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f05ff7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f05ff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f062524e640 5 monclient: authenticate success, global_id 4631 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f062524e640 10 monclient: _renew_subs 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f062524e640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f062524e640 1 -- 10.20.193.90:0/3632194069 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f062013aa80 con 0x7f0620159a70 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f062524e640 10 client.4631.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f062524e640 10 monclient: _renew_subs 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f062524e640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f062524e640 1 -- 10.20.193.90:0/3632194069 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0620146a10 con 0x7f0620159a70 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7f062524e640 1 librados: init done 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1972080348 0 0) 0x7fab3c1fef50 con 0x7fab3c0ba350 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 >> v1:10.20.193.93:6789/0 conn(0x7fab3c0b9f80 legacy=0x7fab3c1fe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 >> v1:10.20.193.90:6789/0 conn(0x7fab3c0585e0 legacy=0x7fab3c15daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab2a7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab2a7fc640 1 monclient: found mon.c 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab2a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab2a7fc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3c160b90 con 0x7fab3c0ba350 2026-02-25T15:01:44.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab2a7fc640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab2a7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab2a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab44144640 5 monclient: authenticate success, global_id 4619 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab44144640 10 monclient: _renew_subs 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab44144640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab44144640 1 -- 10.20.193.90:0/2218227676 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fab3c1611c0 con 0x7fab3c0ba350 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab44144640 10 client.4619.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab44144640 10 monclient: _renew_subs 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab44144640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab44144640 1 -- 10.20.193.90:0/2218227676 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fab3c1ff340 con 0x7fab3c0ba350 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.508+0000 7fab44144640 1 librados: init done 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.509+0000 7f062524e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.509+0000 7f062524e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.509+0000 7f062524e640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.509+0000 7f062524e640 1 -- 10.20.193.90:0/3632194069 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0620057040 con 0x7f0620159a70 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.509+0000 7fab44144640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.509+0000 7fab44144640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.509+0000 7fab44144640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.509+0000 7fab44144640 1 -- 10.20.193.90:0/2218227676 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab3c160dd0 con 0x7fab3c0ba350 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 820011922 0 0) 0x7fa47c003610 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 >> v1:10.20.193.94:6789/0 conn(0x7fa4ac05a010 legacy=0x7fa4ac1fb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 >> v1:10.20.193.90:6789/0 conn(0x7fa4ac1598e0 legacy=0x7fa4ac162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa49a7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa49a7fc640 1 monclient: found mon.b 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa49a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa49a7fc640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ac133b00 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa49a7fc640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa49a7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa49a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa4b3c16640 5 monclient: authenticate success, global_id 4597 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa4b3c16640 10 monclient: _renew_subs 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa4b3c16640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa4b3c16640 1 -- 10.20.193.90:0/2662844873 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ac133de0 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa4b3c16640 10 client.4597.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa4b3c16640 10 monclient: _renew_subs 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa4b3c16640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa4b3c16640 1 -- 10.20.193.90:0/2662844873 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa4ac1fbfc0 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.514+0000 7fa4b3c16640 1 librados: init done 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.515+0000 7fa4b3c16640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.515+0000 7fa4b3c16640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.515+0000 7fa4b3c16640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.515+0000 7fa4b3c16640 1 -- 10.20.193.90:0/2662844873 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4ac0b2f70 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.535+0000 7ff74858a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.535+0000 7ff74858a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.535+0000 7ff74858a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.535+0000 7ff74858a640 1 -- 10.20.193.90:0/54761343 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff740056980 con 0x7ff740056090 2026-02-25T15:01:44.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.559+0000 7f0ca75a2640 10 monclient: get_monmap_and_config 2026-02-25T15:01:44.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.559+0000 7f0ca75a2640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.559+0000 7f0ca75a2640 10 monclient: monmap: 2026-02-25T15:01:44.561 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.561 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.561 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:44.560618+0000 2026-02-25T15:01:44.561 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:44.560618+0000 2026-02-25T15:01:44.561 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.561 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.561 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.561 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.561 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.561 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.560+0000 7f0ca75a2640 1 Processor -- start 2026-02-25T15:01:44.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.560+0000 7f0ca75a2640 1 -- start start 2026-02-25T15:01:44.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.560+0000 7f0ca75a2640 10 monclient: init 2026-02-25T15:01:44.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.561+0000 7f0ca75a2640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.561+0000 7f0ca75a2640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:44.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.561+0000 7f0ca75a2640 10 monclient(hunting): picked mon.noname-c con 0x7f0ca00b9f80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.561+0000 7f0ca75a2640 10 monclient(hunting): picked mon.noname-b con 0x7f0ca00ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.561+0000 7f0ca75a2640 10 monclient(hunting): picked mon.noname-a con 0x7f0ca00585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.561+0000 7f0ca75a2640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca0163930 con 0x7f0ca00585e0 2026-02-25T15:01:44.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.561+0000 7f0ca75a2640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca0163bb0 con 0x7f0ca00ba350 2026-02-25T15:01:44.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.561+0000 7f0ca75a2640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca0163e90 con 0x7f0ca00b9f80 2026-02-25T15:01:44.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.561+0000 7f0ca75a2640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.561+0000 7f0ca75a2640 10 monclient(hunting): authenticate will time out at 2994.161406s 2026-02-25T15:01:44.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.600+0000 7f7650d68640 1 -- 10.20.193.90:0/3614549764 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258681650 0 0) 0x7f763c002910 con 0x7f764c0b9f80 2026-02-25T15:01:44.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.600+0000 7f7650d68640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.600+0000 7f7650d68640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.600+0000 7f7650d68640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.600+0000 7f7650d68640 10 monclient(hunting): my global_id is 4620 2026-02-25T15:01:44.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.600+0000 7f7650d68640 1 -- 10.20.193.90:0/3614549764 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f764c163bb0 con 0x7f764c0b9f80 2026-02-25T15:01:44.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.601+0000 7ff726ffd640 1 -- 10.20.193.90:0/54761343 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (unknown 1274345170 0 1257367751) 0x7ff73005ee40 con 0x7ff740056090 2026-02-25T15:01:44.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.601+0000 7ff726ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-25T15:01:44.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.601+0000 7ff726ffd640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.602 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:44.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.601+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb860003a80 con 0x7fb864159e40 2026-02-25T15:01:44.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.601+0000 7fb8597fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.601+0000 7fb85affd640 4 set_mon_vals no callback set 2026-02-25T15:01:44.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.601+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0608002f90 con 0x7f0620159a70 2026-02-25T15:01:44.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.601+0000 7f05ff7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.601+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0608003ad0 con 0x7f0620159a70 2026-02-25T15:01:44.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.601+0000 7f05ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.601+0000 7f05ff7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:44.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.601+0000 7f061d573640 4 set_mon_vals no callback set 2026-02-25T15:01:44.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.601+0000 7f05ff7fe640 10 monclient: dump: 2026-02-25T15:01:44.603 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.604 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.604 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.604 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.604 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.604 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.604 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.604 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.604 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.604 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.602+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f0608003ce0 con 0x7f0620159a70 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7650d68640 1 -- 10.20.193.90:0/3614549764 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3150872393 0 0) 0x7f7634003610 con 0x7f764c0585e0 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7650d68640 1 -- 10.20.193.90:0/3614549764 >> v1:10.20.193.94:6789/0 conn(0x7f764c0ba350 legacy=0x7f764c057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7650d68640 1 -- 10.20.193.90:0/3614549764 >> v1:10.20.193.90:6789/0 conn(0x7f764c0b9f80 legacy=0x7f764c163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7650d68640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7650d68640 1 monclient: found mon.noname-b 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7650d68640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7650d68640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7650d68640 1 -- 10.20.193.90:0/3614549764 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c0573f0 con 0x7f764c0585e0 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7650d68640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7650d68640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7650d68640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7650d68640 1 -- 10.20.193.90:0/3614549764 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7648003170 con 0x7f764c0585e0 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7650d68640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7650d68640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7650d68640 10 monclient: dump: 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.605 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.606 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.606 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.606 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.606 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.606 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.606 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.606 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7653ff5640 5 monclient: authenticate success, global_id 4603 2026-02-25T15:01:44.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f7653ff5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f061ed76640 10 monclient: get_auth_request con 0x7f05ec079f70 auth_method 0 2026-02-25T15:01:44.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7f05ff7fe640 10 client.4631.objecter ms_handle_connect 0x7f05ec079f70 2026-02-25T15:01:44.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7ff74858a640 10 librados: watch_flush enter 2026-02-25T15:01:44.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7ff74858a640 10 librados: watch_flush exit 2026-02-25T15:01:44.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7ff74858a640 20 client.4587.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7ff74858a640 10 client.4587.objecter successfully canceled tick 2026-02-25T15:01:44.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7ff74858a640 1 -- 10.20.193.90:0/54761343 >> v1:10.20.193.189:6800/3647375982 conn(0x7ff710079f60 legacy=0x7ff71009a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7ff74858a640 10 monclient: shutdown 2026-02-25T15:01:44.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7ff74858a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7ff74858a640 1 -- 10.20.193.90:0/54761343 >> v1:10.20.193.90:6789/0 conn(0x7ff740056090 legacy=0x7ff7401fc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7ff74858a640 1 -- 10.20.193.90:0/54761343 shutdown_connections 2026-02-25T15:01:44.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7ff74858a640 1 -- 10.20.193.90:0/54761343 >> 10.20.193.90:0/54761343 conn(0x7ff7400b78c0 msgr2=0x7ff7400b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7ff74858a640 1 -- 10.20.193.90:0/54761343 shutdown_connections 2026-02-25T15:01:44.608 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7ff74858a640 1 -- 10.20.193.90:0/54761343 wait complete. 2026-02-25T15:01:44.608 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.603+0000 7ff74858a640 1 librados: shutdown 2026-02-25T15:01:44.608 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.604+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f0608095b90 con 0x7f0620159a70 2026-02-25T15:01:44.608 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.604+0000 7f05ff7fe640 10 client.4631.objecter ms_dispatch2 0x7f0620000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:44.608 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.604+0000 7f05ff7fe640 3 client.4631.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:44.608 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.604+0000 7f05ff7fe640 3 client.4631.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:44.608 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.604+0000 7f05ff7fe640 20 client.4631.objecter dump_active .. 0 homeless 2026-02-25T15:01:44.608 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.605+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe7c80048d0 con 0x7fe7e0157b50 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.605+0000 7fe7d57fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.605+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe7c8004ce0 con 0x7fe7e0157b50 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.605+0000 7fe7d57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.605+0000 7fe7d57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.605+0000 7fe7d6ffd640 4 set_mon_vals no callback set 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.605+0000 7fe7d57fa640 10 monclient: dump: 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.605+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fe7c8004ec0 con 0x7fe7e0157b50 2026-02-25T15:01:44.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.606+0000 7fe7e49af640 10 monclient: get_auth_request con 0x7fe7b0079f50 auth_method 0 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.606+0000 7fe7d57fa640 10 client.4608.objecter ms_handle_connect 0x7fe7b0079f50 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.606+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb860003e50 con 0x7fb864159e40 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.606+0000 7fb8597fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.606+0000 7fb8597fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.606+0000 7fb8597fa640 10 monclient: dump: 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.607+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa4a0002ff0 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.607+0000 7fa49a7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.607+0000 7fa49bfff640 4 set_mon_vals no callback set 2026-02-25T15:01:44.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.607+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fb860004070 con 0x7fb864159e40 2026-02-25T15:01:44.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.608+0000 7fb868c56640 10 monclient: get_auth_request con 0x7fb83407a090 auth_method 0 2026-02-25T15:01:44.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.608+0000 7fb8597fa640 10 client.4591.objecter ms_handle_connect 0x7fb83407a090 2026-02-25T15:01:44.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.608+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7fb86005b040 con 0x7fb864159e40 2026-02-25T15:01:44.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.608+0000 7fb8597fa640 10 client.4591.objecter ms_dispatch2 0x7fb864000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:44.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.608+0000 7fb8597fa640 3 client.4591.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:44.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.608+0000 7fb8597fa640 3 client.4591.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:44.611 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.609+0000 7fb8597fa640 20 client.4591.objecter dump_active .. 0 homeless 2026-02-25T15:01:44.612 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.610+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7fe7c8095c50 con 0x7fe7e0157b50 2026-02-25T15:01:44.612 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.610+0000 7fe7d57fa640 10 client.4608.objecter ms_dispatch2 0x7fe7e0000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:44.612 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.610+0000 7fe7d57fa640 3 client.4608.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:44.612 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.610+0000 7fe7d57fa640 3 client.4608.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:44.612 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.610+0000 7fe7d57fa640 20 client.4608.objecter dump_active .. 0 homeless 2026-02-25T15:01:44.612 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.610+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fe7e00b0ac0 con 0x7fe7e0157b50 2026-02-25T15:01:44.612 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.610+0000 7fe7d57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.612 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.610+0000 7fe7d57fa640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:44.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.612+0000 7f0ca5b18640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f0ca00585e0 0x7f0ca0166cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57436/0 (socket says 10.20.193.90:57436) 2026-02-25T15:01:44.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.612+0000 7f0ca5b18640 1 -- 10.20.193.90:0/3365661917 learned_addr learned my addr 10.20.193.90:0/3365661917 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.614 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.6 2026-02-25T15:01:44.614 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:44.614 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.613+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fb86005f7b0 con 0x7fb864159e40 2026-02-25T15:01:44.614 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.613+0000 7fb8597fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.613+0000 7fb8597fa640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:44.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.614+0000 7f0c8ffff640 1 -- 10.20.193.90:0/3365661917 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 109380042 0 0) 0x7f0c9c002c90 con 0x7f0ca00585e0 2026-02-25T15:01:44.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.614+0000 7f0c8ffff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.614+0000 7f0c8ffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.614+0000 7f0c8ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.614+0000 7f0c8ffff640 10 monclient(hunting): my global_id is 4623 2026-02-25T15:01:44.615 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.614+0000 7f0c8ffff640 1 -- 10.20.193.90:0/3365661917 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c88003610 con 0x7f0ca00585e0 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.615+0000 7f0c8ffff640 1 -- 10.20.193.90:0/3365661917 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 632527717 0 0) 0x7f0c88003610 con 0x7f0ca00585e0 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.615+0000 7f0c8ffff640 1 -- 10.20.193.90:0/3365661917 >> v1:10.20.193.94:6789/0 conn(0x7f0ca00b9f80 legacy=0x7f0ca0163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 1 -- 10.20.193.90:0/3365661917 >> v1:10.20.193.93:6789/0 conn(0x7f0ca00ba350 legacy=0x7f0ca0057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 1 monclient: found mon.noname-a 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 1 -- 10.20.193.90:0/3365661917 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca00573f0 con 0x7f0ca00585e0 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 1 -- 10.20.193.90:0/3365661917 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0c9c003340 con 0x7f0ca00585e0 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 10 monclient: dump: 2026-02-25T15:01:44.617 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.618 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.618 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.618 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.618 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.618 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.618 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.618 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.618 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.618 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0ca75a2640 5 monclient: authenticate success, global_id 4623 2026-02-25T15:01:44.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0ca75a2640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 1 -- 10.20.193.90:0/3365661917 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0c9c003550 con 0x7f0ca00585e0 2026-02-25T15:01:44.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 1 -- 10.20.193.90:0/3365661917 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0c9c004d10 con 0x7f0ca00585e0 2026-02-25T15:01:44.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:44.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0c8ffff640 10 monclient: dump: 2026-02-25T15:01:44.619 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.619 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.619 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.619 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.619 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.619 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.619 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.619 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0ca75a2640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.616+0000 7f0ca75a2640 4 set_mon_vals no callback set 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.617+0000 7f0ca75a2640 10 monclient: shutdown 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.617+0000 7f0ca75a2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.617+0000 7f0ca75a2640 1 -- 10.20.193.90:0/3365661917 >> v1:10.20.193.90:6789/0 conn(0x7f0ca00585e0 legacy=0x7f0ca0166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.617+0000 7f0ca75a2640 1 -- 10.20.193.90:0/3365661917 shutdown_connections 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.617+0000 7f0ca75a2640 1 -- 10.20.193.90:0/3365661917 >> 10.20.193.90:0/3365661917 conn(0x7f0ca00b7990 msgr2=0x7f0ca00b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.617+0000 7f0ca75a2640 1 -- 10.20.193.90:0/3365661917 shutdown_connections 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.617+0000 7f0ca75a2640 1 -- 10.20.193.90:0/3365661917 wait complete. 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.617+0000 7f0ca75a2640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.617+0000 7f0ca75a2640 10 monclient: monmap: 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.620 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.621 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 1 librados: starting msgr at 2026-02-25T15:01:44.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 1 librados: starting objecter 2026-02-25T15:01:44.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:44.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 1 Processor -- start 2026-02-25T15:01:44.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 1 -- start start 2026-02-25T15:01:44.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 1 librados: setting wanted keys 2026-02-25T15:01:44.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 1 librados: calling monclient init 2026-02-25T15:01:44.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 10 monclient: init 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 10 monclient(hunting): picked mon.noname-a con 0x7f0ca00585e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 10 monclient(hunting): picked mon.noname-b con 0x7f0ca00b9f80 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 10 monclient(hunting): picked mon.noname-c con 0x7f0ca00ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca01febd0 con 0x7f0ca00585e0 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca00b3100 con 0x7f0ca00b9f80 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca00b33e0 con 0x7f0ca00ba350 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca75a2640 10 monclient(hunting): authenticate will time out at 2994.218575s 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca5317640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f0ca00585e0 0x7f0ca00b2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57450/0 (socket says 10.20.193.90:57450) 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.618+0000 7f0ca5317640 1 -- 10.20.193.90:0/4091173096 learned_addr learned my addr 10.20.193.90:0/4091173096 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.619+0000 7f0c8dffb640 10 client.?.objecter ms_handle_connect 0x7f0ca00585e0 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.619+0000 7f0c8dffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.620+0000 7f0c8dffb640 1 -- 10.20.193.90:0/4091173096 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1232443947 0 0) 0x7f0ca01febd0 con 0x7f0ca00585e0 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.620+0000 7f0c8dffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.620+0000 7f0c8dffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.620+0000 7f0c8dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.620+0000 7f0c8dffb640 10 monclient(hunting): my global_id is 4626 2026-02-25T15:01:44.622 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.620+0000 7f0c8dffb640 1 -- 10.20.193.90:0/4091173096 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c70003610 con 0x7f0ca00585e0 2026-02-25T15:01:44.623 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.622+0000 7f0c8dffb640 1 -- 10.20.193.90:0/4091173096 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 691059223 0 0) 0x7f0c70003610 con 0x7f0ca00585e0 2026-02-25T15:01:44.623 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.622+0000 7f0c8dffb640 1 -- 10.20.193.90:0/4091173096 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0ca01febd0 con 0x7f0ca00585e0 2026-02-25T15:01:44.623 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.622+0000 7f0c8dffb640 1 -- 10.20.193.90:0/4091173096 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0c90002fd0 con 0x7f0ca00585e0 2026-02-25T15:01:44.623 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.622+0000 7f0c8dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.623 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.622+0000 7f0c8dffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:44.623 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.622+0000 7f0c8dffb640 10 monclient(hunting): dump: 2026-02-25T15:01:44.623 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.623 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.624 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.624 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.624 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.624 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.624 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.624 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.624 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.624 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.624 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.622+0000 7f0c8dffb640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:44.625 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0c8dffb640 1 -- 10.20.193.90:0/4091173096 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3077397787 0 0) 0x7f0ca01febd0 con 0x7f0ca00585e0 2026-02-25T15:01:44.625 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0c8dffb640 1 -- 10.20.193.90:0/4091173096 >> v1:10.20.193.94:6789/0 conn(0x7f0ca00ba350 legacy=0x7f0ca0161a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.625 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0c8dffb640 1 -- 10.20.193.90:0/4091173096 >> v1:10.20.193.93:6789/0 conn(0x7f0ca00b9f80 legacy=0x7f0ca01fe4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.625 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0c8dffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.625 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0c8dffb640 1 monclient: found mon.a 2026-02-25T15:01:44.625 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0c8dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.625 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0c8dffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.625 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0c8dffb640 1 -- 10.20.193.90:0/4091173096 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca00b36c0 con 0x7f0ca00585e0 2026-02-25T15:01:44.625 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0c8dffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.625 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0c8dffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.625 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0c8dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.625 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0ca75a2640 5 monclient: authenticate success, global_id 4626 2026-02-25T15:01:44.625 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0ca75a2640 10 monclient: _renew_subs 2026-02-25T15:01:44.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0ca75a2640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0ca75a2640 1 -- 10.20.193.90:0/4091173096 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca0161270 con 0x7f0ca00585e0 2026-02-25T15:01:44.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0ca75a2640 10 client.4626.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:44.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0ca75a2640 10 monclient: _renew_subs 2026-02-25T15:01:44.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0ca75a2640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0ca75a2640 1 -- 10.20.193.90:0/4091173096 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0ca0162180 con 0x7f0ca00585e0 2026-02-25T15:01:44.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0ca75a2640 1 librados: init done 2026-02-25T15:01:44.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0ca75a2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0ca75a2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0ca75a2640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.624+0000 7f0ca75a2640 1 -- 10.20.193.90:0/4091173096 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca00b64c0 con 0x7f0ca00585e0 2026-02-25T15:01:44.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.627+0000 7feb112a6640 10 monclient: get_monmap_and_config 2026-02-25T15:01:44.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.627+0000 7feb112a6640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.627+0000 7feb112a6640 10 monclient: monmap: 2026-02-25T15:01:44.628 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.628 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.629 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:44.628425+0000 2026-02-25T15:01:44.629 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:44.628425+0000 2026-02-25T15:01:44.629 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.629 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.629 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.629 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.629 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.629 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.628+0000 7feb112a6640 1 Processor -- start 2026-02-25T15:01:44.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.628+0000 7feb112a6640 1 -- start start 2026-02-25T15:01:44.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.628+0000 7feb112a6640 10 monclient: init 2026-02-25T15:01:44.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.629+0000 7feb112a6640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.629+0000 7feb112a6640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:44.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.629+0000 7feb112a6640 10 monclient(hunting): picked mon.noname-a con 0x7feb0c0b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.629+0000 7feb112a6640 10 monclient(hunting): picked mon.noname-b con 0x7feb0c0ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.629+0000 7feb112a6640 10 monclient(hunting): picked mon.noname-c con 0x7feb0c0585e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.629+0000 7feb112a6640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb0c163930 con 0x7feb0c0b9f80 2026-02-25T15:01:44.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.629+0000 7feb112a6640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb0c163bb0 con 0x7feb0c0ba350 2026-02-25T15:01:44.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.629+0000 7feb112a6640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb0c163e90 con 0x7feb0c0585e0 2026-02-25T15:01:44.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.629+0000 7feb112a6640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.629+0000 7feb112a6640 10 monclient(hunting): authenticate will time out at 2994.229208s 2026-02-25T15:01:44.634 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.633+0000 7f0c8dffb640 1 -- 10.20.193.90:0/4091173096 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0c900049f0 con 0x7f0ca00585e0 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.633+0000 7f0c8dffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.633+0000 7f0c8dffb640 1 -- 10.20.193.90:0/4091173096 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0c90004ba0 con 0x7f0ca00585e0 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.633+0000 7f0c8dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.633+0000 7f0c8dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.633+0000 7f0c8f7fe640 4 set_mon_vals no callback set 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.633+0000 7f0c8dffb640 10 monclient: dump: 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.635 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.636 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.634+0000 7f0c8dffb640 1 -- 10.20.193.90:0/4091173096 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f0c9001e080 con 0x7f0ca00585e0 2026-02-25T15:01:44.636 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.635+0000 7f0ca4b16640 10 monclient: get_auth_request con 0x7f0c70079f30 auth_method 0 2026-02-25T15:01:44.636 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.635+0000 7f0c8dffb640 10 client.4626.objecter ms_handle_connect 0x7f0c70079f30 2026-02-25T15:01:44.638 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.637+0000 7f0c8dffb640 1 -- 10.20.193.90:0/4091173096 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f0c90095d40 con 0x7f0ca00585e0 2026-02-25T15:01:44.638 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.637+0000 7f0c8dffb640 10 client.4626.objecter ms_dispatch2 0x7f0ca0000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:44.638 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.637+0000 7f0c8dffb640 3 client.4626.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:44.638 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.637+0000 7f0c8dffb640 3 client.4626.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:44.639 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.637+0000 7feb0affd640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7feb0c0b9f80 0x7feb0c163220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57458/0 (socket says 10.20.193.90:57458) 2026-02-25T15:01:44.639 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.637+0000 7feb0affd640 1 -- 10.20.193.90:0/3495433107 learned_addr learned my addr 10.20.193.90:0/3495433107 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.639 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.637+0000 7f0c8dffb640 20 client.4626.objecter dump_active .. 0 homeless 2026-02-25T15:01:44.691 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.690+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f0608095e20 con 0x7f0620159a70 2026-02-25T15:01:44.691 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.690+0000 7f05ff7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.691 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.690+0000 7f05ff7fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:44.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.695+0000 7f7650d68640 1 -- 10.20.193.90:0/3614549764 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7648003380 con 0x7f764c0585e0 2026-02-25T15:01:44.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.695+0000 7f7650d68640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.695+0000 7f7653ff5640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:44.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.695+0000 7f7653ff5640 4 set_mon_vals no callback set 2026-02-25T15:01:44.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.696+0000 7f7653ff5640 10 monclient: shutdown 2026-02-25T15:01:44.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.696+0000 7f7653ff5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.696+0000 7f7653ff5640 1 -- 10.20.193.90:0/3614549764 >> v1:10.20.193.93:6789/0 conn(0x7f764c0585e0 legacy=0x7f764c166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.696+0000 7f7653ff5640 1 -- 10.20.193.90:0/3614549764 shutdown_connections 2026-02-25T15:01:44.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.696+0000 7f7653ff5640 1 -- 10.20.193.90:0/3614549764 >> 10.20.193.90:0/3614549764 conn(0x7f764c0b7990 msgr2=0x7f764c0b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.696+0000 7f7653ff5640 1 -- 10.20.193.90:0/3614549764 shutdown_connections 2026-02-25T15:01:44.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.696+0000 7f7653ff5640 1 -- 10.20.193.90:0/3614549764 wait complete. 2026-02-25T15:01:44.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.696+0000 7f7653ff5640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 10 monclient: monmap: 2026-02-25T15:01:44.697 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.697 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:44.697 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 1 librados: starting msgr at 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 1 librados: starting objecter 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fab380032c0 con 0x7fab3c0ba350 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7fab2a7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab38003ec0 con 0x7fab3c0ba350 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7fab2a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7fab2a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7fab2a7fc640 10 monclient: dump: 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7fab2bfff640 4 set_mon_vals no callback set 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 1 Processor -- start 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 1 -- start start 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 1 librados: setting wanted keys 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 1 librados: calling monclient init 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 10 monclient: init 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 10 monclient(hunting): picked mon.noname-b con 0x7f764c0585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 10 monclient(hunting): picked mon.noname-c con 0x7f764c0b9f80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 10 monclient(hunting): picked mon.noname-a con 0x7f764c0ba350 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f764c1581a0 con 0x7f764c0ba350 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f764c157100 con 0x7f764c0585e0 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f764c1573e0 con 0x7f764c0b9f80 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.697+0000 7f7653ff5640 10 monclient(hunting): authenticate will time out at 2994.297817s 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.698+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fab380040a0 con 0x7fab3c0ba350 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.698+0000 7fab41eb9640 10 monclient: get_auth_request con 0x7fab0c07a220 auth_method 0 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.698+0000 7fab2a7fc640 10 client.4619.objecter ms_handle_connect 0x7fab0c07a220 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.698+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7fab38095d30 con 0x7fab3c0ba350 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.698+0000 7fab2a7fc640 10 client.4619.objecter ms_dispatch2 0x7fab3c000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.698+0000 7fab2a7fc640 3 client.4619.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:44.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.698+0000 7fab2a7fc640 3 client.4619.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:44.700 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.699+0000 7fab2a7fc640 20 client.4619.objecter dump_active .. 0 homeless 2026-02-25T15:01:44.700 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.699+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa4a0003a50 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.700 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.699+0000 7fa49a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.700 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.699+0000 7fa49a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:44.700 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.699+0000 7fa49a7fc640 10 monclient: dump: 2026-02-25T15:01:44.700 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.700 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.700 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.700 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.700 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.700 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.700 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.700 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.700 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.700 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.700+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fa4a001cf30 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.700+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fab3c160dd0 con 0x7fab3c0ba350 2026-02-25T15:01:44.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.700+0000 7fab2a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.700+0000 7fab2a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:44.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.700+0000 7fa4b198b640 10 monclient: get_auth_request con 0x7fa47c079f00 auth_method 0 2026-02-25T15:01:44.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.700+0000 7fa49a7fc640 10 client.4597.objecter ms_handle_connect 0x7fa47c079f00 2026-02-25T15:01:44.702 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.701+0000 7f7651569640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f764c0b9f80 0x7f764c15f660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45884/0 (socket says 10.20.193.90:45884) 2026-02-25T15:01:44.702 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.701+0000 7f7651569640 1 -- 10.20.193.90:0/2039975139 learned_addr learned my addr 10.20.193.90:0/2039975139 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.702 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.701+0000 7f763a7fc640 10 client.?.objecter ms_handle_connect 0x7f764c0b9f80 2026-02-25T15:01:44.702 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.701+0000 7f763a7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.702 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.702+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7fa4a0059c40 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.702 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.702+0000 7fa49a7fc640 10 client.4597.objecter ms_dispatch2 0x7fa4ac000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:44.702 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.702+0000 7fa49a7fc640 3 client.4597.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:44.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.702+0000 7fa49a7fc640 3 client.4597.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:44.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.702+0000 7fa49a7fc640 20 client.4597.objecter dump_active .. 0 homeless 2026-02-25T15:01:44.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.702+0000 7feb09ffb640 1 -- 10.20.193.90:0/3495433107 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3371032323 0 0) 0x7feb0c163e90 con 0x7feb0c0585e0 2026-02-25T15:01:44.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.702+0000 7feb09ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.702+0000 7feb09ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.702+0000 7feb09ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.702+0000 7feb09ffb640 10 monclient(hunting): my global_id is 4637 2026-02-25T15:01:44.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.702+0000 7feb09ffb640 1 -- 10.20.193.90:0/3495433107 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feaf8003610 con 0x7feb0c0585e0 2026-02-25T15:01:44.704 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.703+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fa4a005e3b0 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.704 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.703+0000 7fa49a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.704 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.703+0000 7fa49a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:44.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.704+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3709196097 0 0) 0x7f764c1573e0 con 0x7f764c0b9f80 2026-02-25T15:01:44.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.704+0000 7f763a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.704+0000 7f763a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.704+0000 7f763a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.704+0000 7f763a7fc640 10 monclient(hunting): my global_id is 4640 2026-02-25T15:01:44.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.704+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f761c003610 con 0x7f764c0b9f80 2026-02-25T15:01:44.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb09ffb640 1 -- 10.20.193.90:0/3495433107 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2194088725 0 0) 0x7feaf8003610 con 0x7feb0c0585e0 2026-02-25T15:01:44.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb09ffb640 1 -- 10.20.193.90:0/3495433107 >> v1:10.20.193.93:6789/0 conn(0x7feb0c0ba350 legacy=0x7feb0c057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb09ffb640 1 -- 10.20.193.90:0/3495433107 >> v1:10.20.193.90:6789/0 conn(0x7feb0c0b9f80 legacy=0x7feb0c163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb09ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb09ffb640 1 monclient: found mon.noname-c 2026-02-25T15:01:44.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb09ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb09ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb09ffb640 1 -- 10.20.193.90:0/3495433107 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c0573f0 con 0x7feb0c0585e0 2026-02-25T15:01:44.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb09ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb09ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb09ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb09ffb640 1 -- 10.20.193.90:0/3495433107 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7feafc003170 con 0x7feb0c0585e0 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb09ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb09ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb09ffb640 10 monclient: dump: 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb112a6640 5 monclient: authenticate success, global_id 4637 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.705+0000 7feb112a6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.706+0000 7f763a7fc640 10 client.?.objecter ms_handle_connect 0x7f764c0585e0 2026-02-25T15:01:44.707 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.706+0000 7f763a7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1641664904 0 0) 0x7f761c003610 con 0x7f764c0b9f80 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f764c1573e0 con 0x7f764c0b9f80 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7640003060 con 0x7f764c0b9f80 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7f763a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7f763a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7f763a7fc640 10 monclient(hunting): dump: 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7f763a7fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2994060515 0 0) 0x7f764c157100 con 0x7f764c0585e0 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7f763a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7f763a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7f763a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7f763a7fc640 10 monclient(hunting): my global_id is 4609 2026-02-25T15:01:44.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f761c003610 con 0x7f764c0585e0 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7feb09ffb640 1 -- 10.20.193.90:0/3495433107 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7feafc003380 con 0x7feb0c0585e0 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7feb09ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7feb112a6640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7feb112a6640 4 set_mon_vals no callback set 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7feb09ffb640 1 -- 10.20.193.90:0/3495433107 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7feafc0037c0 con 0x7feb0c0585e0 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7feb09ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7feb09ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.708+0000 7feb09ffb640 10 monclient: dump: 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 10 monclient: shutdown 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 1 -- 10.20.193.90:0/3495433107 >> v1:10.20.193.94:6789/0 conn(0x7feb0c0585e0 legacy=0x7feb0c166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 1 -- 10.20.193.90:0/3495433107 shutdown_connections 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 1 -- 10.20.193.90:0/3495433107 >> 10.20.193.90:0/3495433107 conn(0x7feb0c0b7990 msgr2=0x7feb0c0b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 1 -- 10.20.193.90:0/3495433107 shutdown_connections 2026-02-25T15:01:44.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 1 -- 10.20.193.90:0/3495433107 wait complete. 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 10 monclient: build_initial_monmap 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 10 monclient: monmap: 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 1 librados: starting msgr at 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 1 librados: starting objecter 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 1 Processor -- start 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 1 -- start start 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 1 librados: setting wanted keys 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 1 librados: calling monclient init 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.709+0000 7feb112a6640 10 monclient: init 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7feb112a6640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7feb112a6640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7feb112a6640 10 monclient(hunting): picked mon.noname-b con 0x7feb0c0585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:44.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7feb112a6640 10 monclient(hunting): picked mon.noname-c con 0x7feb0c0b9f80 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7feb112a6640 10 monclient(hunting): picked mon.noname-a con 0x7feb0c0ba350 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7feb112a6640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb0c1fe0c0 con 0x7feb0c0ba350 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7feb112a6640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb0c1fd260 con 0x7feb0c0585e0 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7feb112a6640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb0c1fd540 con 0x7feb0c0b9f80 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7feb112a6640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7feb112a6640 10 monclient(hunting): authenticate will time out at 2994.310376s 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1092948106 0 0) 0x7f764c1573e0 con 0x7f764c0b9f80 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 >> v1:10.20.193.93:6789/0 conn(0x7f764c0585e0 legacy=0x7f764c15dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 >> v1:10.20.193.90:6789/0 conn(0x7f764c0ba350 legacy=0x7f764c13a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f763a7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f763a7fc640 1 monclient: found mon.c 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f763a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f763a7fc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c1576c0 con 0x7f764c0b9f80 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f763a7fc640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f763a7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f763a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f7653ff5640 5 monclient: authenticate success, global_id 4640 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f7653ff5640 10 monclient: _renew_subs 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f7653ff5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f7653ff5640 1 -- 10.20.193.90:0/2039975139 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f764c1579a0 con 0x7f764c0b9f80 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f7653ff5640 10 client.4640.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f7653ff5640 10 monclient: _renew_subs 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f7653ff5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f7653ff5640 1 -- 10.20.193.90:0/2039975139 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f764c15fd70 con 0x7f764c0b9f80 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.710+0000 7f7653ff5640 1 librados: init done 2026-02-25T15:01:44.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.711+0000 7f7653ff5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.711+0000 7f7653ff5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.711+0000 7f7653ff5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.711+0000 7f7653ff5640 1 -- 10.20.193.90:0/2039975139 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f764c0b6c80 con 0x7f764c0b9f80 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.711+0000 7feb0affd640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7feb0c0585e0 0x7feb0c160790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:51226/0 (socket says 10.20.193.90:51226) 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.711+0000 7feb0affd640 1 -- 10.20.193.90:0/3729500060 learned_addr learned my addr 10.20.193.90:0/3729500060 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.711+0000 7feaeb7fe640 10 client.?.objecter ms_handle_connect 0x7feb0c0585e0 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.711+0000 7feaeb7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.711+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7640003740 con 0x7f764c0b9f80 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.711+0000 7f763a7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.711+0000 7f763bfff640 4 set_mon_vals no callback set 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.712+0000 7feaeb7fe640 1 -- 10.20.193.90:0/3729500060 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3467002578 0 0) 0x7feb0c1fd260 con 0x7feb0c0585e0 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.712+0000 7feaeb7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.712+0000 7feaeb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.712+0000 7feaeb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.712+0000 7feaeb7fe640 10 monclient(hunting): my global_id is 4612 2026-02-25T15:01:44.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.712+0000 7feaeb7fe640 1 -- 10.20.193.90:0/3729500060 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fead8003610 con 0x7feb0c0585e0 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.715+0000 7feaeb7fe640 1 -- 10.20.193.90:0/3729500060 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3080393136 0 0) 0x7fead8003610 con 0x7feb0c0585e0 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.715+0000 7feaeb7fe640 1 -- 10.20.193.90:0/3729500060 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7feb0c1fd260 con 0x7feb0c0585e0 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.715+0000 7feaeb7fe640 1 -- 10.20.193.90:0/3729500060 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7feb00003010 con 0x7feb0c0585e0 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.715+0000 7feaeb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.715+0000 7feaeb7fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.715+0000 7feaeb7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.715+0000 7feaeb7fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.715+0000 7feaeb7fe640 10 client.?.objecter ms_handle_connect 0x7feb0c0b9f80 2026-02-25T15:01:44.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.715+0000 7feaeb7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feaeb7fe640 1 -- 10.20.193.90:0/3729500060 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 706317691 0 0) 0x7feb0c1fd260 con 0x7feb0c0585e0 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feaeb7fe640 1 -- 10.20.193.90:0/3729500060 >> v1:10.20.193.94:6789/0 conn(0x7feb0c0b9f80 legacy=0x7feb0c1f9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feaeb7fe640 1 -- 10.20.193.90:0/3729500060 >> v1:10.20.193.90:6789/0 conn(0x7feb0c0ba350 legacy=0x7feb0c162130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feaeb7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feaeb7fe640 1 monclient: found mon.b 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feaeb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feaeb7fe640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feaeb7fe640 1 -- 10.20.193.90:0/3729500060 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c1fd820 con 0x7feb0c0585e0 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feaeb7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feaeb7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feaeb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feb112a6640 5 monclient: authenticate success, global_id 4612 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feb112a6640 10 monclient: _renew_subs 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feb112a6640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feb112a6640 1 -- 10.20.193.90:0/3729500060 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7feb0c1fdb00 con 0x7feb0c0585e0 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feb112a6640 10 client.4612.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feb112a6640 10 monclient: _renew_subs 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feb112a6640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feb112a6640 1 -- 10.20.193.90:0/3729500060 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7feb0c1f9740 con 0x7feb0c0585e0 2026-02-25T15:01:44.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.717+0000 7feb112a6640 1 librados: init done 2026-02-25T15:01:44.719 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.718+0000 7feb112a6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.719 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.718+0000 7feb112a6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.719 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.718+0000 7feb112a6640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.719 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.718+0000 7feb112a6640 1 -- 10.20.193.90:0/3729500060 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb0c1f99d0 con 0x7feb0c0585e0 2026-02-25T15:01:44.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.719+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7640003b50 con 0x7f764c0b9f80 2026-02-25T15:01:44.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.719+0000 7f763a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.719+0000 7f763a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:44.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.719+0000 7f763a7fc640 10 monclient: dump: 2026-02-25T15:01:44.720 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.720 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.720 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.720 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.720 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.720 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.720 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.720 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.720 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.720 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.721 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.720+0000 7fe7e6c3a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-25T15:01:44.721 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.720+0000 7fe7e6c3a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-25T15:01:44.721 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.720+0000 7fe7e6c3a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.721 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.720+0000 7fe7e6c3a640 1 -- 10.20.193.90:0/2051178685 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe7e00b0850 con 0x7fe7e0157b50 2026-02-25T15:01:44.721 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.720+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f7640003d70 con 0x7f764c0b9f80 2026-02-25T15:01:44.721 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.721+0000 7f7651d6a640 10 monclient: get_auth_request con 0x7f761c079f50 auth_method 0 2026-02-25T15:01:44.722 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.721+0000 7f763a7fc640 10 client.4640.objecter ms_handle_connect 0x7f761c079f50 2026-02-25T15:01:44.724 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.724+0000 7feaeb7fe640 1 -- 10.20.193.90:0/3729500060 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7feb000048d0 con 0x7feb0c0585e0 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.724+0000 7feaeb7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.724+0000 7feaeb7fe640 1 -- 10.20.193.90:0/3729500060 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7feb00004b40 con 0x7feb0c0585e0 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.724+0000 7feaeb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.724+0000 7feaeb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.724+0000 7feb097fa640 4 set_mon_vals no callback set 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.724+0000 7feaeb7fe640 10 monclient: dump: 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.724+0000 7f0c8dffb640 1 -- 10.20.193.90:0/4091173096 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f0c9005f6a0 con 0x7f0ca00585e0 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.724+0000 7f0c8dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.725 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.724+0000 7f0c8dffb640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:44.726 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.726+0000 7fb86aee1640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-25T15:01:44.726 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.726+0000 7fb86aee1640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-25T15:01:44.726 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.726+0000 7fb86aee1640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.727 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.726+0000 7fb86aee1640 1 -- 10.20.193.90:0/67494453 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb864159220 con 0x7fb864159e40 2026-02-25T15:01:44.727 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.726+0000 7feaeb7fe640 1 -- 10.20.193.90:0/3729500060 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7feb00004ce0 con 0x7feb0c0585e0 2026-02-25T15:01:44.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.727+0000 7feb0a7fc640 10 monclient: get_auth_request con 0x7fead8079e00 auth_method 0 2026-02-25T15:01:44.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.727+0000 7feaeb7fe640 10 client.4612.objecter ms_handle_connect 0x7fead8079e00 2026-02-25T15:01:44.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.727+0000 7feaeb7fe640 1 -- 10.20.193.90:0/3729500060 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7feb00095d70 con 0x7feb0c0585e0 2026-02-25T15:01:44.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.727+0000 7feaeb7fe640 10 client.4612.objecter ms_dispatch2 0x7feb0c000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:44.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.727+0000 7feaeb7fe640 3 client.4612.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:44.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.727+0000 7feaeb7fe640 3 client.4612.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:44.729 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.728+0000 7feaeb7fe640 20 client.4612.objecter dump_active .. 0 homeless 2026-02-25T15:01:44.729 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.728+0000 7fe7d57fa640 1 -- 10.20.193.90:0/2051178685 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (unknown 383520633 0 521551722) 0x7fe7e00b0850 con 0x7fe7e0157b50 2026-02-25T15:01:44.729 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.728+0000 7fe7d57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-25T15:01:44.729 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.728+0000 7fe7d57fa640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.729 INFO:teuthology.orchestra.run.trial090.stdout:94489280516 2026-02-25T15:01:44.731 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.730+0000 7fe7e6c3a640 10 librados: watch_flush enter 2026-02-25T15:01:44.731 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.730+0000 7fe7e6c3a640 10 librados: watch_flush exit 2026-02-25T15:01:44.731 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.730+0000 7fe7e6c3a640 20 client.4608.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.731 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.730+0000 7fe7e6c3a640 10 client.4608.objecter successfully canceled tick 2026-02-25T15:01:44.731 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.730+0000 7fe7e6c3a640 1 -- 10.20.193.90:0/2051178685 >> v1:10.20.193.189:6800/3647375982 conn(0x7fe7b0079f50 legacy=0x7fe7b009a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.732 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.730+0000 7fe7e6c3a640 10 monclient: shutdown 2026-02-25T15:01:44.732 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.730+0000 7fe7e6c3a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.732 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.730+0000 7fe7e6c3a640 1 -- 10.20.193.90:0/2051178685 >> v1:10.20.193.90:6789/0 conn(0x7fe7e0157b50 legacy=0x7fe7e01226b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.732 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.730+0000 7fe7e6c3a640 1 -- 10.20.193.90:0/2051178685 shutdown_connections 2026-02-25T15:01:44.732 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.730+0000 7fe7e6c3a640 1 -- 10.20.193.90:0/2051178685 >> 10.20.193.90:0/2051178685 conn(0x7fe7e00b2ef0 msgr2=0x7fe7e00ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.732 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.730+0000 7fe7e6c3a640 1 -- 10.20.193.90:0/2051178685 shutdown_connections 2026-02-25T15:01:44.732 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.730+0000 7fe7e6c3a640 1 -- 10.20.193.90:0/2051178685 wait complete. 2026-02-25T15:01:44.732 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.730+0000 7fe7e6c3a640 1 librados: shutdown 2026-02-25T15:01:44.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.3 2026-02-25T15:01:44.741 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:44.804 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.802+0000 7f062524e640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-25T15:01:44.804 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.802+0000 7f062524e640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-25T15:01:44.804 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.802+0000 7f062524e640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.804 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.802+0000 7f062524e640 1 -- 10.20.193.90:0/3632194069 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0620164080 con 0x7f0620159a70 2026-02-25T15:01:44.806 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.805+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f7640095c30 con 0x7f764c0b9f80 2026-02-25T15:01:44.806 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.805+0000 7f763a7fc640 10 client.4640.objecter ms_dispatch2 0x7f764c000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:44.807 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.805+0000 7f763a7fc640 3 client.4640.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:44.807 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.805+0000 7f763a7fc640 3 client.4640.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:44.807 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.805+0000 7f763a7fc640 20 client.4640.objecter dump_active .. 0 homeless 2026-02-25T15:01:44.808 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.807+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f764005f590 con 0x7f764c0b9f80 2026-02-25T15:01:44.808 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.807+0000 7f763a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.808 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.807+0000 7f763a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:44.812 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.810+0000 7fab44144640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-25T15:01:44.812 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.810+0000 7fab44144640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-25T15:01:44.812 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.810+0000 7fab44144640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.812 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.810+0000 7fab44144640 1 -- 10.20.193.90:0/2218227676 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 14} v 0) -- 0x7fab3c1572e0 con 0x7fab3c0ba350 2026-02-25T15:01:44.812 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.811+0000 7f05ff7fe640 1 -- 10.20.193.90:0/3632194069 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (unknown 2131975755 0 1257367751) 0x7f060805f4f0 con 0x7f0620159a70 2026-02-25T15:01:44.812 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.811+0000 7f05ff7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-25T15:01:44.812 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.811+0000 7f05ff7fe640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.813 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:44.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.813+0000 7f062524e640 10 librados: watch_flush enter 2026-02-25T15:01:44.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.813+0000 7f062524e640 10 librados: watch_flush exit 2026-02-25T15:01:44.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.813+0000 7f062524e640 20 client.4631.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.813+0000 7f062524e640 10 client.4631.objecter successfully canceled tick 2026-02-25T15:01:44.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.813+0000 7f062524e640 1 -- 10.20.193.90:0/3632194069 >> v1:10.20.193.189:6800/3647375982 conn(0x7f05ec079f70 legacy=0x7f05ec09a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.813+0000 7f062524e640 10 monclient: shutdown 2026-02-25T15:01:44.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.813+0000 7f062524e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.813+0000 7f062524e640 1 -- 10.20.193.90:0/3632194069 >> v1:10.20.193.94:6789/0 conn(0x7f0620159a70 legacy=0x7f062015b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.813+0000 7f062524e640 1 -- 10.20.193.90:0/3632194069 shutdown_connections 2026-02-25T15:01:44.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.813+0000 7f062524e640 1 -- 10.20.193.90:0/3632194069 >> 10.20.193.90:0/3632194069 conn(0x7f06200b7780 msgr2=0x7f0620162de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.813+0000 7f062524e640 1 -- 10.20.193.90:0/3632194069 shutdown_connections 2026-02-25T15:01:44.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.813+0000 7f062524e640 1 -- 10.20.193.90:0/3632194069 wait complete. 2026-02-25T15:01:44.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.813+0000 7f062524e640 1 librados: shutdown 2026-02-25T15:01:44.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.814+0000 7fb8597fa640 1 -- 10.20.193.90:0/67494453 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (unknown 92182286 0 1257367751) 0x7fb86005f0b0 con 0x7fb864159e40 2026-02-25T15:01:44.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.814+0000 7fb8597fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-25T15:01:44.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.814+0000 7fb8597fa640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.817 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:44.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.814+0000 7fab2a7fc640 1 -- 10.20.193.90:0/2218227676 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 14}]=0 v0) ==== 75+0+12 (unknown 2547163239 0 1257367751) 0x7fab3805f690 con 0x7fab3c0ba350 2026-02-25T15:01:44.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.814+0000 7fab2a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-25T15:01:44.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.814+0000 7fab2a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.818 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:44.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.816+0000 7feaeb7fe640 1 -- 10.20.193.90:0/3729500060 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7feb0c163930 con 0x7feb0c0585e0 2026-02-25T15:01:44.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.816+0000 7feaeb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:44.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.816+0000 7feaeb7fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:44.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fb86aee1640 10 librados: watch_flush enter 2026-02-25T15:01:44.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fb86aee1640 10 librados: watch_flush exit 2026-02-25T15:01:44.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fb86aee1640 20 client.4591.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fb86aee1640 10 client.4591.objecter successfully canceled tick 2026-02-25T15:01:44.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fb86aee1640 1 -- 10.20.193.90:0/67494453 >> v1:10.20.193.189:6800/3647375982 conn(0x7fb83407a090 legacy=0x7fb83409a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fb86aee1640 10 monclient: shutdown 2026-02-25T15:01:44.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fb86aee1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fb86aee1640 1 -- 10.20.193.90:0/67494453 >> v1:10.20.193.93:6789/0 conn(0x7fb864159e40 legacy=0x7fb8641fc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fb86aee1640 1 -- 10.20.193.90:0/67494453 shutdown_connections 2026-02-25T15:01:44.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fb86aee1640 1 -- 10.20.193.90:0/67494453 >> 10.20.193.90:0/67494453 conn(0x7fb8640b7780 msgr2=0x7fb8641582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fb86aee1640 1 -- 10.20.193.90:0/67494453 shutdown_connections 2026-02-25T15:01:44.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fb86aee1640 1 -- 10.20.193.90:0/67494453 wait complete. 2026-02-25T15:01:44.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fb86aee1640 1 librados: shutdown 2026-02-25T15:01:44.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fab44144640 10 librados: watch_flush enter 2026-02-25T15:01:44.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fab44144640 10 librados: watch_flush exit 2026-02-25T15:01:44.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fab44144640 20 client.4619.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fab44144640 10 client.4619.objecter successfully canceled tick 2026-02-25T15:01:44.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fab44144640 1 -- 10.20.193.90:0/2218227676 >> v1:10.20.193.189:6800/3647375982 conn(0x7fab0c07a220 legacy=0x7fab0c09a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fab44144640 10 monclient: shutdown 2026-02-25T15:01:44.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fab44144640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fab44144640 1 -- 10.20.193.90:0/2218227676 >> v1:10.20.193.94:6789/0 conn(0x7fab3c0ba350 legacy=0x7fab3c1619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.817+0000 7fab44144640 1 -- 10.20.193.90:0/2218227676 shutdown_connections 2026-02-25T15:01:44.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.818+0000 7fab44144640 1 -- 10.20.193.90:0/2218227676 >> 10.20.193.90:0/2218227676 conn(0x7fab3c0b7990 msgr2=0x7fab3c15acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.818+0000 7fab44144640 1 -- 10.20.193.90:0/2218227676 shutdown_connections 2026-02-25T15:01:44.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.818+0000 7fab44144640 1 -- 10.20.193.90:0/2218227676 wait complete. 2026-02-25T15:01:44.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.818+0000 7fab44144640 1 librados: shutdown 2026-02-25T15:01:44.824 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.818+0000 7fa4b3c16640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-25T15:01:44.824 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.818+0000 7fa4b3c16640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-25T15:01:44.824 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.818+0000 7fa4b3c16640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.824 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.818+0000 7fa4b3c16640 1 -- 10.20.193.90:0/2662844873 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fa4ac15d350 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.824 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.821+0000 7fa49a7fc640 1 -- 10.20.193.90:0/2662844873 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+12 (unknown 3405404620 0 1257367751) 0x7fa4a005dcb0 con 0x7fa4ac0b27f0 2026-02-25T15:01:44.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.821+0000 7fa49a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-25T15:01:44.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.821+0000 7fa49a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.825 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:44.826 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.5 2026-02-25T15:01:44.827 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:44.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.824+0000 7fa4b3c16640 10 librados: watch_flush enter 2026-02-25T15:01:44.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.824+0000 7fa4b3c16640 10 librados: watch_flush exit 2026-02-25T15:01:44.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.824+0000 7fa4b3c16640 20 client.4597.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.824+0000 7fa4b3c16640 10 client.4597.objecter successfully canceled tick 2026-02-25T15:01:44.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.824+0000 7fa4b3c16640 1 -- 10.20.193.90:0/2662844873 >> v1:10.20.193.189:6800/3647375982 conn(0x7fa47c079f00 legacy=0x7fa47c09a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.824+0000 7fa4b3c16640 10 monclient: shutdown 2026-02-25T15:01:44.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.824+0000 7fa4b3c16640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.824+0000 7fa4b3c16640 1 -- 10.20.193.90:0/2662844873 >> v1:10.20.193.93:6789/0 conn(0x7fa4ac0b27f0 legacy=0x7fa4ac1345e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.824+0000 7fa4b3c16640 1 -- 10.20.193.90:0/2662844873 shutdown_connections 2026-02-25T15:01:44.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.824+0000 7fa4b3c16640 1 -- 10.20.193.90:0/2662844873 >> 10.20.193.90:0/2662844873 conn(0x7fa4ac0b7120 msgr2=0x7fa4ac0b74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.824+0000 7fa4b3c16640 1 -- 10.20.193.90:0/2662844873 shutdown_connections 2026-02-25T15:01:44.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.824+0000 7fa4b3c16640 1 -- 10.20.193.90:0/2662844873 wait complete. 2026-02-25T15:01:44.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.824+0000 7fa4b3c16640 1 librados: shutdown 2026-02-25T15:01:44.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.14 2026-02-25T15:01:44.831 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:44.831 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.2 2026-02-25T15:01:44.831 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:44.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.11 2026-02-25T15:01:44.836 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:44.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.835+0000 7f0ca75a2640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-25T15:01:44.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.835+0000 7f0ca75a2640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-25T15:01:44.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.835+0000 7f0ca75a2640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:44.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.835+0000 7f0ca75a2640 1 -- 10.20.193.90:0/4091173096 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 15} v 0) -- 0x7f0ca00b91d0 con 0x7f0ca00585e0 2026-02-25T15:01:44.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.840+0000 7f0c8dffb640 1 -- 10.20.193.90:0/4091173096 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 15}]=0 v0) ==== 75+0+12 (unknown 2221946896 0 1257367751) 0x7f0c9005efa0 con 0x7f0ca00585e0 2026-02-25T15:01:44.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.840+0000 7f0c8dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-25T15:01:44.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.840+0000 7f0c8dffb640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.842 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:44.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.843+0000 7f0ca75a2640 10 librados: watch_flush enter 2026-02-25T15:01:44.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.843+0000 7f0ca75a2640 10 librados: watch_flush exit 2026-02-25T15:01:44.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.843+0000 7f0ca75a2640 20 client.4626.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.843+0000 7f0ca75a2640 10 client.4626.objecter successfully canceled tick 2026-02-25T15:01:44.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.843+0000 7f0ca75a2640 1 -- 10.20.193.90:0/4091173096 >> v1:10.20.193.189:6800/3647375982 conn(0x7f0c70079f30 legacy=0x7f0c7009a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.843+0000 7f0ca75a2640 10 monclient: shutdown 2026-02-25T15:01:44.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.843+0000 7f0ca75a2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.843+0000 7f0ca75a2640 1 -- 10.20.193.90:0/4091173096 >> v1:10.20.193.90:6789/0 conn(0x7f0ca00585e0 legacy=0x7f0ca00b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.843+0000 7f0ca75a2640 1 -- 10.20.193.90:0/4091173096 shutdown_connections 2026-02-25T15:01:44.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.843+0000 7f0ca75a2640 1 -- 10.20.193.90:0/4091173096 >> 10.20.193.90:0/4091173096 conn(0x7f0ca00b7990 msgr2=0x7f0ca0158a10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.843+0000 7f0ca75a2640 1 -- 10.20.193.90:0/4091173096 shutdown_connections 2026-02-25T15:01:44.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.843+0000 7f0ca75a2640 1 -- 10.20.193.90:0/4091173096 wait complete. 2026-02-25T15:01:44.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.843+0000 7f0ca75a2640 1 librados: shutdown 2026-02-25T15:01:44.853 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.15 2026-02-25T15:01:44.853 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:44.918 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.916+0000 7f7653ff5640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-25T15:01:44.918 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.916+0000 7f7653ff5640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-25T15:01:44.918 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.916+0000 7f7653ff5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:44.918 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.916+0000 7f7653ff5640 1 -- 10.20.193.90:0/2039975139 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 13} v 0) -- 0x7f764c056cf0 con 0x7f764c0b9f80 2026-02-25T15:01:44.919 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.918+0000 7f763a7fc640 1 -- 10.20.193.90:0/2039975139 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 13}]=0 v0) ==== 75+0+12 (unknown 3988744482 0 521551722) 0x7f764005ee90 con 0x7f764c0b9f80 2026-02-25T15:01:44.920 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.918+0000 7f763a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-25T15:01:44.920 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.918+0000 7f763a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.920 INFO:teuthology.orchestra.run.trial090.stdout:94489280516 2026-02-25T15:01:44.922 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.920+0000 7f7653ff5640 10 librados: watch_flush enter 2026-02-25T15:01:44.922 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.920+0000 7f7653ff5640 10 librados: watch_flush exit 2026-02-25T15:01:44.922 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.920+0000 7f7653ff5640 20 client.4640.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.922 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.920+0000 7f7653ff5640 10 client.4640.objecter successfully canceled tick 2026-02-25T15:01:44.922 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.920+0000 7f7653ff5640 1 -- 10.20.193.90:0/2039975139 >> v1:10.20.193.189:6800/3647375982 conn(0x7f761c079f50 legacy=0x7f761c09a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.922 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.920+0000 7f7653ff5640 10 monclient: shutdown 2026-02-25T15:01:44.922 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.920+0000 7f7653ff5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.922 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.920+0000 7f7653ff5640 1 -- 10.20.193.90:0/2039975139 >> v1:10.20.193.94:6789/0 conn(0x7f764c0b9f80 legacy=0x7f764c15f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.922 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.920+0000 7f7653ff5640 1 -- 10.20.193.90:0/2039975139 shutdown_connections 2026-02-25T15:01:44.922 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.920+0000 7f7653ff5640 1 -- 10.20.193.90:0/2039975139 >> 10.20.193.90:0/2039975139 conn(0x7f764c0b7990 msgr2=0x7f764c13b930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.922 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.920+0000 7f7653ff5640 1 -- 10.20.193.90:0/2039975139 shutdown_connections 2026-02-25T15:01:44.922 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.920+0000 7f7653ff5640 1 -- 10.20.193.90:0/2039975139 wait complete. 2026-02-25T15:01:44.922 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.920+0000 7f7653ff5640 1 librados: shutdown 2026-02-25T15:01:44.931 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.13 2026-02-25T15:01:44.931 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.929+0000 7feb112a6640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-25T15:01:44.931 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.929+0000 7feb112a6640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-25T15:01:44.931 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.929+0000 7feb112a6640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:44.931 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.929+0000 7feb112a6640 1 -- 10.20.193.90:0/3729500060 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7feb0c0b6c80 con 0x7feb0c0585e0 2026-02-25T15:01:44.931 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:44.932 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.931+0000 7feaeb7fe640 1 -- 10.20.193.90:0/3729500060 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (unknown 1823589948 0 1257367751) 0x7feb0005f6d0 con 0x7feb0c0585e0 2026-02-25T15:01:44.932 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.931+0000 7feaeb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-25T15:01:44.932 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.931+0000 7feaeb7fe640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:44.933 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:44.935 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.933+0000 7feb112a6640 10 librados: watch_flush enter 2026-02-25T15:01:44.935 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.933+0000 7feb112a6640 10 librados: watch_flush exit 2026-02-25T15:01:44.935 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.933+0000 7feb112a6640 20 client.4612.objecter shutdown clearing up homeless session... 2026-02-25T15:01:44.935 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.933+0000 7feb112a6640 10 client.4612.objecter successfully canceled tick 2026-02-25T15:01:44.935 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.933+0000 7feb112a6640 1 -- 10.20.193.90:0/3729500060 >> v1:10.20.193.189:6800/3647375982 conn(0x7fead8079e00 legacy=0x7fead809a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.936 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.933+0000 7feb112a6640 10 monclient: shutdown 2026-02-25T15:01:44.936 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.933+0000 7feb112a6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:44.936 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.933+0000 7feb112a6640 1 -- 10.20.193.90:0/3729500060 >> v1:10.20.193.93:6789/0 conn(0x7feb0c0585e0 legacy=0x7feb0c160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:44.936 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.933+0000 7feb112a6640 1 -- 10.20.193.90:0/3729500060 shutdown_connections 2026-02-25T15:01:44.936 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.933+0000 7feb112a6640 1 -- 10.20.193.90:0/3729500060 >> 10.20.193.90:0/3729500060 conn(0x7feb0c0b7990 msgr2=0x7feb0c15bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:44.936 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.933+0000 7feb112a6640 1 -- 10.20.193.90:0/3729500060 shutdown_connections 2026-02-25T15:01:44.936 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.933+0000 7feb112a6640 1 -- 10.20.193.90:0/3729500060 wait complete. 2026-02-25T15:01:44.936 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:44.933+0000 7feb112a6640 1 librados: shutdown 2026-02-25T15:01:44.943 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.4 2026-02-25T15:01:44.943 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:45.094 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-25T15:01:45.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.159+0000 7f4961ee1640 10 monclient: get_monmap_and_config 2026-02-25T15:01:45.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.159+0000 7f4961ee1640 10 monclient: build_initial_monmap 2026-02-25T15:01:45.160 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.159+0000 7f4961ee1640 10 monclient: monmap: 2026-02-25T15:01:45.160 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:45.160 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.160 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:45.160029+0000 2026-02-25T15:01:45.160 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:45.160029+0000 2026-02-25T15:01:45.160 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:45.160 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.160 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:45.160 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:45.160 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:45.160 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.159+0000 7f4961ee1640 1 Processor -- start 2026-02-25T15:01:45.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.159+0000 7f4961ee1640 1 -- start start 2026-02-25T15:01:45.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.159+0000 7f4961ee1640 10 monclient: init 2026-02-25T15:01:45.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.160+0000 7f4961ee1640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:45.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.160+0000 7f4961ee1640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:45.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.160+0000 7f4961ee1640 10 monclient(hunting): picked mon.noname-c con 0x7f495c15d070 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:45.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.160+0000 7f4961ee1640 10 monclient(hunting): picked mon.noname-b con 0x7f495c15d440 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:45.162 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.160+0000 7f4961ee1640 10 monclient(hunting): picked mon.noname-a con 0x7f495c166f60 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:45.162 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.160+0000 7f4961ee1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f495c164e60 con 0x7f495c166f60 2026-02-25T15:01:45.162 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.160+0000 7f4961ee1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f495c0b1790 con 0x7f495c15d440 2026-02-25T15:01:45.162 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.160+0000 7f4961ee1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f495c0b1a70 con 0x7f495c15d070 2026-02-25T15:01:45.162 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.160+0000 7f4961ee1640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:45.162 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.160+0000 7f4961ee1640 10 monclient(hunting): authenticate will time out at 2994.760482s 2026-02-25T15:01:45.162 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.160+0000 7f495affd640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f495c15d440 0x7f495c166850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:51236/0 (socket says 10.20.193.90:51236) 2026-02-25T15:01:45.162 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.160+0000 7f495affd640 1 -- 10.20.193.90:0/2865266229 learned_addr learned my addr 10.20.193.90:0/2865266229 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:45.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 1 -- 10.20.193.90:0/2865266229 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 497620395 0 0) 0x7f495c0b1a70 con 0x7f495c15d070 2026-02-25T15:01:45.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 10 monclient(hunting): my global_id is 4646 2026-02-25T15:01:45.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 1 -- 10.20.193.90:0/2865266229 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4940003610 con 0x7f495c15d070 2026-02-25T15:01:45.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 1 -- 10.20.193.90:0/2865266229 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 617628886 0 0) 0x7f495c0b1790 con 0x7f495c15d440 2026-02-25T15:01:45.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 10 monclient(hunting): my global_id is 4615 2026-02-25T15:01:45.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 1 -- 10.20.193.90:0/2865266229 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f495c0b1a70 con 0x7f495c15d440 2026-02-25T15:01:45.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 1 -- 10.20.193.90:0/2865266229 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4021332625 0 0) 0x7f495c164e60 con 0x7f495c166f60 2026-02-25T15:01:45.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 10 monclient(hunting): my global_id is 4632 2026-02-25T15:01:45.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.162+0000 7f495a7fc640 1 -- 10.20.193.90:0/2865266229 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f495c0b1790 con 0x7f495c166f60 2026-02-25T15:01:45.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f495a7fc640 1 -- 10.20.193.90:0/2865266229 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3029315574 0 0) 0x7f4940003610 con 0x7f495c15d070 2026-02-25T15:01:45.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f495a7fc640 1 -- 10.20.193.90:0/2865266229 >> v1:10.20.193.93:6789/0 conn(0x7f495c15d440 legacy=0x7f495c166850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f495a7fc640 1 -- 10.20.193.90:0/2865266229 >> v1:10.20.193.90:6789/0 conn(0x7f495c166f60 legacy=0x7f495c0b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f495a7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:01:45.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f495a7fc640 1 monclient: found mon.noname-c 2026-02-25T15:01:45.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f495a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:45.165 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f495a7fc640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f495a7fc640 1 -- 10.20.193.90:0/2865266229 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c0b1d50 con 0x7f495c15d070 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f495a7fc640 10 monclient: _finish_auth 0 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f495a7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f495a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f495a7fc640 1 -- 10.20.193.90:0/2865266229 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4948002df0 con 0x7f495c15d070 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f495a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f495a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f495a7fc640 10 monclient: dump: 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f4961ee1640 5 monclient: authenticate success, global_id 4646 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.164+0000 7f4961ee1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.165+0000 7f495a7fc640 1 -- 10.20.193.90:0/2865266229 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4948002fc0 con 0x7f495c15d070 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.165+0000 7f495a7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:45.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.165+0000 7f495a7fc640 1 -- 10.20.193.90:0/2865266229 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4948003400 con 0x7f495c15d070 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.165+0000 7f4961ee1640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.165+0000 7f4961ee1640 4 set_mon_vals no callback set 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.165+0000 7f495a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.165+0000 7f495a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.165+0000 7f495a7fc640 10 monclient: dump: 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.165+0000 7f4961ee1640 10 monclient: shutdown 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.165+0000 7f4961ee1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.165+0000 7f4961ee1640 1 -- 10.20.193.90:0/2865266229 >> v1:10.20.193.94:6789/0 conn(0x7f495c15d070 legacy=0x7f495c164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.165+0000 7f4961ee1640 1 -- 10.20.193.90:0/2865266229 shutdown_connections 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.165+0000 7f4961ee1640 1 -- 10.20.193.90:0/2865266229 >> 10.20.193.90:0/2865266229 conn(0x7f495c0b2ef0 msgr2=0x7f495c0b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.165+0000 7f4961ee1640 1 -- 10.20.193.90:0/2865266229 shutdown_connections 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.165+0000 7f4961ee1640 1 -- 10.20.193.90:0/2865266229 wait complete. 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 10 monclient: build_initial_monmap 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 10 monclient: monmap: 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:45.167 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 1 librados: starting msgr at 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 1 librados: starting objecter 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 1 Processor -- start 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 1 -- start start 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 1 librados: setting wanted keys 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 1 librados: calling monclient init 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 10 monclient: init 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 10 monclient(hunting): picked mon.noname-a con 0x7f495c15d070 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 10 monclient(hunting): picked mon.noname-b con 0x7f495c15d440 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 10 monclient(hunting): picked mon.noname-c con 0x7f495c166f60 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f495c0b5aa0 con 0x7f495c15d070 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f495c13d7a0 con 0x7f495c15d440 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f495c13da80 con 0x7f495c166f60 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f4961ee1640 10 monclient(hunting): authenticate will time out at 2994.766707s 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f495b7fe640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f495c15d070 0x7f495c0b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57494/0 (socket says 10.20.193.90:57494) 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.166+0000 7f495b7fe640 1 -- 10.20.193.90:0/200253457 learned_addr learned my addr 10.20.193.90:0/200253457 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.167+0000 7f4937fff640 10 client.?.objecter ms_handle_connect 0x7f495c166f60 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.167+0000 7f4937fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.167+0000 7f4937fff640 10 client.?.objecter ms_handle_connect 0x7f495c15d070 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.167+0000 7f4937fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.167+0000 7f4937fff640 10 client.?.objecter ms_handle_connect 0x7f495c15d440 2026-02-25T15:01:45.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.167+0000 7f4937fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2042405494 0 0) 0x7f495c13d7a0 con 0x7f495c15d440 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 10 monclient(hunting): my global_id is 4618 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f492c003610 con 0x7f495c15d440 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2219315076 0 0) 0x7f495c13da80 con 0x7f495c166f60 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 10 monclient(hunting): my global_id is 4649 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f495c13d7a0 con 0x7f495c166f60 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3436341659 0 0) 0x7f495c0b5aa0 con 0x7f495c15d070 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 10 monclient(hunting): my global_id is 4635 2026-02-25T15:01:45.169 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.168+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f495c13da80 con 0x7f495c15d070 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2507818094 0 0) 0x7f495c13d7a0 con 0x7f495c166f60 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f495c0b5aa0 con 0x7f495c166f60 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4950003330 con 0x7f495c166f60 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 10 monclient(hunting): dump: 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2739236723 0 0) 0x7f492c003610 con 0x7f495c15d440 2026-02-25T15:01:45.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f495c13d7a0 con 0x7f495c15d440 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f494c003040 con 0x7f495c15d440 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 10 monclient(hunting): dump: 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4223030826 0 0) 0x7f495c13da80 con 0x7f495c15d070 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f492c003610 con 0x7f495c15d070 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4948003310 con 0x7f495c15d070 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.171+0000 7f4937fff640 10 monclient(hunting): dump: 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.173 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 280609386 0 0) 0x7f495c0b5aa0 con 0x7f495c166f60 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 >> v1:10.20.193.93:6789/0 conn(0x7f495c15d440 legacy=0x7f495c126a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 >> v1:10.20.193.90:6789/0 conn(0x7f495c15d070 legacy=0x7f495c0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4937fff640 10 monclient: _finish_hunting 0 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4937fff640 1 monclient: found mon.c 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4937fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4937fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c13dd60 con 0x7f495c166f60 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4937fff640 10 monclient: _finish_auth 0 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4937fff640 10 monclient: _check_auth_tickets 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4937fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4961ee1640 5 monclient: authenticate success, global_id 4649 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4961ee1640 10 monclient: _renew_subs 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4961ee1640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4961ee1640 1 -- 10.20.193.90:0/200253457 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f495c13e040 con 0x7f495c166f60 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4961ee1640 10 client.4649.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4961ee1640 10 monclient: _renew_subs 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4961ee1640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4961ee1640 1 -- 10.20.193.90:0/200253457 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f495c126500 con 0x7f495c166f60 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4961ee1640 1 librados: init done 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4961ee1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4961ee1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4961ee1640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:45.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.173+0000 7f4961ee1640 1 -- 10.20.193.90:0/200253457 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495c05dbb0 con 0x7f495c166f60 2026-02-25T15:01:45.181 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-25T15:01:45.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.181+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4950003a80 con 0x7f495c166f60 2026-02-25T15:01:45.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.181+0000 7f4937fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:45.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.181+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4950003dd0 con 0x7f495c166f60 2026-02-25T15:01:45.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.181+0000 7f4937fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.181+0000 7f4937fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:45.182 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.181+0000 7f4937fff640 10 monclient: dump: 2026-02-25T15:01:45.182 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.182 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.182 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.182 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.182 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.182 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.183 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.183 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.183 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.183 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.181+0000 7f4959ffb640 4 set_mon_vals no callback set 2026-02-25T15:01:45.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.182+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f495001d2b0 con 0x7f495c166f60 2026-02-25T15:01:45.183 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.183+0000 7f495b7fe640 10 monclient: get_auth_request con 0x7f492c07a0f0 auth_method 0 2026-02-25T15:01:45.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.183+0000 7f4937fff640 10 client.4649.objecter ms_handle_connect 0x7f492c07a0f0 2026-02-25T15:01:45.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.183+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f4950094b00 con 0x7f495c166f60 2026-02-25T15:01:45.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.183+0000 7f4937fff640 10 client.4649.objecter ms_dispatch2 0x7f495c000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:45.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.183+0000 7f4937fff640 3 client.4649.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:45.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.183+0000 7f4937fff640 3 client.4649.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:45.184 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.183+0000 7f4937fff640 20 client.4649.objecter dump_active .. 0 homeless 2026-02-25T15:01:45.223 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.248+0000 7f38615fa640 10 monclient: get_monmap_and_config 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.248+0000 7f38615fa640 10 monclient: build_initial_monmap 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.248+0000 7f38615fa640 10 monclient: monmap: 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:45.249031+0000 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:45.249031+0000 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.248+0000 7f38615fa640 1 Processor -- start 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.248+0000 7f38615fa640 1 -- start start 2026-02-25T15:01:45.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.248+0000 7f38615fa640 10 monclient: init 2026-02-25T15:01:45.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.249+0000 7f38615fa640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:45.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.249+0000 7f38615fa640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:45.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.249+0000 7f38615fa640 10 monclient(hunting): picked mon.noname-c con 0x7f385c157780 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:45.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.249+0000 7f38615fa640 10 monclient(hunting): picked mon.noname-a con 0x7f385c157b50 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:45.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.249+0000 7f38615fa640 10 monclient(hunting): picked mon.noname-b con 0x7f385c164d00 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:45.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.249+0000 7f38615fa640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f385c05de70 con 0x7f385c157b50 2026-02-25T15:01:45.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.249+0000 7f38615fa640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f385c055b20 con 0x7f385c164d00 2026-02-25T15:01:45.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.249+0000 7f38615fa640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f385c055e00 con 0x7f385c157780 2026-02-25T15:01:45.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.249+0000 7f38615fa640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:45.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.249+0000 7f38615fa640 10 monclient(hunting): authenticate will time out at 2994.849455s 2026-02-25T15:01:45.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.249+0000 7f385a7fc640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f385c157b50 0x7f385c1645f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57496/0 (socket says 10.20.193.90:57496) 2026-02-25T15:01:45.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.249+0000 7f385a7fc640 1 -- 10.20.193.90:0/3135554844 learned_addr learned my addr 10.20.193.90:0/3135554844 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:45.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.251+0000 7f3859ffb640 1 -- 10.20.193.90:0/3135554844 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1028778974 0 0) 0x7f385c05de70 con 0x7f385c157b50 2026-02-25T15:01:45.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.251+0000 7f3859ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.251+0000 7f3859ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.251+0000 7f3859ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.251+0000 7f3859ffb640 10 monclient(hunting): my global_id is 4638 2026-02-25T15:01:45.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.251+0000 7f3859ffb640 1 -- 10.20.193.90:0/3135554844 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3848003610 con 0x7f385c157b50 2026-02-25T15:01:45.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.251+0000 7f3859ffb640 1 -- 10.20.193.90:0/3135554844 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 433823790 0 0) 0x7f385c055b20 con 0x7f385c164d00 2026-02-25T15:01:45.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.251+0000 7f3859ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.251+0000 7f3859ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.251+0000 7f3859ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.251+0000 7f3859ffb640 10 monclient(hunting): my global_id is 4621 2026-02-25T15:01:45.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.251+0000 7f3859ffb640 1 -- 10.20.193.90:0/3135554844 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f385c05de70 con 0x7f385c164d00 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f3859ffb640 1 -- 10.20.193.90:0/3135554844 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3320857436 0 0) 0x7f3848003610 con 0x7f385c157b50 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f3859ffb640 1 -- 10.20.193.90:0/3135554844 >> v1:10.20.193.94:6789/0 conn(0x7f385c157780 legacy=0x7f385c056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f3859ffb640 1 -- 10.20.193.90:0/3135554844 >> v1:10.20.193.93:6789/0 conn(0x7f385c164d00 legacy=0x7f385c1c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f3859ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f3859ffb640 1 monclient: found mon.noname-a 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f3859ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f3859ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f3859ffb640 1 -- 10.20.193.90:0/3135554844 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c0560e0 con 0x7f385c157b50 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f3859ffb640 10 monclient: _finish_auth 0 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f3859ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f3859ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f3859ffb640 1 -- 10.20.193.90:0/3135554844 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3844002df0 con 0x7f385c157b50 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f38615fa640 5 monclient: authenticate success, global_id 4638 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f38615fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f3859ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f3859ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f3859ffb640 10 monclient: dump: 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.254 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.253+0000 7f38615fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.254+0000 7f3859ffb640 1 -- 10.20.193.90:0/3135554844 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3844003000 con 0x7f385c157b50 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.254+0000 7f3859ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.254+0000 7f3859ffb640 1 -- 10.20.193.90:0/3135554844 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3844003440 con 0x7f385c157b50 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.254+0000 7f3859ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.254+0000 7f3859ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.254+0000 7f3859ffb640 10 monclient: dump: 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.254+0000 7f38615fa640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.254+0000 7f38615fa640 4 set_mon_vals no callback set 2026-02-25T15:01:45.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.254+0000 7f38615fa640 10 monclient: shutdown 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.254+0000 7f38615fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.254+0000 7f38615fa640 1 -- 10.20.193.90:0/3135554844 >> v1:10.20.193.90:6789/0 conn(0x7f385c157b50 legacy=0x7f385c1645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.254+0000 7f38615fa640 1 -- 10.20.193.90:0/3135554844 shutdown_connections 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.254+0000 7f38615fa640 1 -- 10.20.193.90:0/3135554844 >> 10.20.193.90:0/3135554844 conn(0x7f385c0b2ef0 msgr2=0x7f385c0b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.254+0000 7f38615fa640 1 -- 10.20.193.90:0/3135554844 shutdown_connections 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.254+0000 7f38615fa640 1 -- 10.20.193.90:0/3135554844 wait complete. 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 10 monclient: build_initial_monmap 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 10 monclient: monmap: 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 1 librados: starting msgr at 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 1 librados: starting objecter 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 1 Processor -- start 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 1 -- start start 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 1 librados: setting wanted keys 2026-02-25T15:01:45.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 1 librados: calling monclient init 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 10 monclient: init 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 10 monclient(hunting): picked mon.noname-b con 0x7f385c157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 10 monclient(hunting): picked mon.noname-c con 0x7f385c157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 10 monclient(hunting): picked mon.noname-a con 0x7f385c164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f385c156f20 con 0x7f385c164d00 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f385c1c6e90 con 0x7f385c157780 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f385c1560c0 con 0x7f385c157b50 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f38615fa640 10 monclient(hunting): authenticate will time out at 2994.855715s 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f385affd640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f385c157780 0x7f385c13aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:51254/0 (socket says 10.20.193.90:51254) 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.255+0000 7f385affd640 1 -- 10.20.193.90:0/1775404871 learned_addr learned my addr 10.20.193.90:0/1775404871 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.256+0000 7f383b7fe640 10 client.?.objecter ms_handle_connect 0x7f385c164d00 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.256+0000 7f383b7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.256+0000 7f383b7fe640 10 client.?.objecter ms_handle_connect 0x7f385c157780 2026-02-25T15:01:45.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.256+0000 7f383b7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.257+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3633262952 0 0) 0x7f385c156f20 con 0x7f385c164d00 2026-02-25T15:01:45.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.257+0000 7f383b7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.257+0000 7f383b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.257+0000 7f383b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.257+0000 7f383b7fe640 10 monclient(hunting): my global_id is 4641 2026-02-25T15:01:45.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.257+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3828003610 con 0x7f385c164d00 2026-02-25T15:01:45.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.257+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2767916482 0 0) 0x7f385c1c6e90 con 0x7f385c157780 2026-02-25T15:01:45.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.257+0000 7f383b7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.257+0000 7f383b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.257+0000 7f383b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.257+0000 7f383b7fe640 10 monclient(hunting): my global_id is 4624 2026-02-25T15:01:45.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.257+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f385c156f20 con 0x7f385c157780 2026-02-25T15:01:45.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.259+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4156324469 0 0) 0x7f3828003610 con 0x7f385c164d00 2026-02-25T15:01:45.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.259+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f385c1c6e90 con 0x7f385c164d00 2026-02-25T15:01:45.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.259+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f384c003370 con 0x7f385c164d00 2026-02-25T15:01:45.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.259+0000 7f383b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:45.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.259+0000 7f383b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.259+0000 7f383b7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.259+0000 7f383b7fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.259+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2915302856 0 0) 0x7f385c156f20 con 0x7f385c157780 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.259+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3828003610 con 0x7f385c157780 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.259+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3850002fd0 con 0x7f385c157780 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.259+0000 7f383b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.259+0000 7f383b7fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.259+0000 7f383b7fe640 10 monclient(hunting): dump: 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.261 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1703450109 0 0) 0x7f3828003610 con 0x7f385c157780 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 >> v1:10.20.193.94:6789/0 conn(0x7f385c157b50 legacy=0x7f385c0abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 >> v1:10.20.193.90:6789/0 conn(0x7f385c164d00 legacy=0x7f385c1c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f383b7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f383b7fe640 1 monclient: found mon.b 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f383b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f383b7fe640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c1563a0 con 0x7f385c157780 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f383b7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f383b7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f383b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f38615fa640 5 monclient: authenticate success, global_id 4624 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f38615fa640 10 monclient: _renew_subs 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f38615fa640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f38615fa640 1 -- 10.20.193.90:0/1775404871 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f385c156680 con 0x7f385c157780 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f38615fa640 10 client.4624.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:45.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f38615fa640 10 monclient: _renew_subs 2026-02-25T15:01:45.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f38615fa640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:45.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f38615fa640 1 -- 10.20.193.90:0/1775404871 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f385c156c40 con 0x7f385c157780 2026-02-25T15:01:45.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.261+0000 7f38615fa640 1 librados: init done 2026-02-25T15:01:45.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.262+0000 7f38615fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.262+0000 7f38615fa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.262+0000 7f38615fa640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:45.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.262+0000 7f38615fa640 1 -- 10.20.193.90:0/1775404871 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f385c058fa0 con 0x7f385c157780 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.270+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f38500035c0 con 0x7f385c157780 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.270+0000 7f383b7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.270+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3850004b90 con 0x7f385c157780 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.270+0000 7f383b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.270+0000 7f383b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.270+0000 7f383b7fe640 10 monclient: dump: 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.270+0000 7f38597fa640 4 set_mon_vals no callback set 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.270+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f495005e460 con 0x7f495c166f60 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.270+0000 7f4937fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.270+0000 7f4937fff640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:45.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.271+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f385001e070 con 0x7f385c157780 2026-02-25T15:01:45.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.271+0000 7f385a7fc640 10 monclient: get_auth_request con 0x7f3828079ea0 auth_method 0 2026-02-25T15:01:45.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.271+0000 7f383b7fe640 10 client.4624.objecter ms_handle_connect 0x7f3828079ea0 2026-02-25T15:01:45.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.272+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f3850095be0 con 0x7f385c157780 2026-02-25T15:01:45.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.272+0000 7f383b7fe640 10 client.4624.objecter ms_dispatch2 0x7f385c000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:45.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.272+0000 7f383b7fe640 3 client.4624.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:45.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.272+0000 7f383b7fe640 3 client.4624.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:45.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.272+0000 7f383b7fe640 20 client.4624.objecter dump_active .. 0 homeless 2026-02-25T15:01:45.286 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-25T15:01:45.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.288+0000 7f7290ee8640 10 monclient: get_monmap_and_config 2026-02-25T15:01:45.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.288+0000 7f7290ee8640 10 monclient: build_initial_monmap 2026-02-25T15:01:45.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.288+0000 7f7290ee8640 10 monclient: monmap: 2026-02-25T15:01:45.289 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:45.289 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.289 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:45.289416+0000 2026-02-25T15:01:45.289 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:45.289416+0000 2026-02-25T15:01:45.289 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:45.289 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.289 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:45.290 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:45.290 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:45.290 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.289+0000 7f7290ee8640 1 Processor -- start 2026-02-25T15:01:45.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.289+0000 7f7290ee8640 1 -- start start 2026-02-25T15:01:45.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.289+0000 7f7290ee8640 10 monclient: init 2026-02-25T15:01:45.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.289+0000 7f7290ee8640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:45.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.289+0000 7f7290ee8640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:01:45.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.289+0000 7f7290ee8640 10 monclient(hunting): picked mon.noname-a con 0x7f728c159da0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:45.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.289+0000 7f7290ee8640 10 monclient(hunting): picked mon.noname-b con 0x7f728c165880 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:45.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.289+0000 7f7290ee8640 10 monclient(hunting): picked mon.noname-c con 0x7f728c056090 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:45.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.289+0000 7f7290ee8640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f728c15a170 con 0x7f728c159da0 2026-02-25T15:01:45.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.289+0000 7f7290ee8640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f728c0b4650 con 0x7f728c165880 2026-02-25T15:01:45.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.289+0000 7f7290ee8640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f728c0b4930 con 0x7f728c056090 2026-02-25T15:01:45.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.289+0000 7f7290ee8640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:45.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.289+0000 7f7290ee8640 10 monclient(hunting): authenticate will time out at 2994.889886s 2026-02-25T15:01:45.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.290+0000 7f728ad76640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f728c056090 0x7f728c0b3770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45950/0 (socket says 10.20.193.90:45950) 2026-02-25T15:01:45.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.290+0000 7f728ad76640 1 -- 10.20.193.90:0/1177831247 learned_addr learned my addr 10.20.193.90:0/1177831247 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:45.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.291+0000 7f7289573640 1 -- 10.20.193.90:0/1177831247 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3031448853 0 0) 0x7f728c0b4930 con 0x7f728c056090 2026-02-25T15:01:45.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.291+0000 7f7289573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.291+0000 7f7289573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.291+0000 7f7289573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.291+0000 7f7289573640 10 monclient(hunting): my global_id is 4652 2026-02-25T15:01:45.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.291+0000 7f7289573640 1 -- 10.20.193.90:0/1177831247 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7278003610 con 0x7f728c056090 2026-02-25T15:01:45.292 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2026-02-25T15:01:45.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7289573640 1 -- 10.20.193.90:0/1177831247 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3051589845 0 0) 0x7f7278003610 con 0x7f728c056090 2026-02-25T15:01:45.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7289573640 1 -- 10.20.193.90:0/1177831247 >> v1:10.20.193.93:6789/0 conn(0x7f728c165880 legacy=0x7f728c055980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T15:01:45.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7289573640 1 -- 10.20.193.90:0/1177831247 >> v1:10.20.193.90:6789/0 conn(0x7f728c159da0 legacy=0x7f728c165170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7289573640 10 monclient: _finish_hunting 0 2026-02-25T15:01:45.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7289573640 1 monclient: found mon.noname-c 2026-02-25T15:01:45.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7289573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:45.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7289573640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:45.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7289573640 1 -- 10.20.193.90:0/1177831247 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c0b4c10 con 0x7f728c056090 2026-02-25T15:01:45.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7289573640 10 monclient: _finish_auth 0 2026-02-25T15:01:45.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7289573640 10 monclient: _check_auth_tickets 2026-02-25T15:01:45.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7289573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7289573640 1 -- 10.20.193.90:0/1177831247 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f727c003170 con 0x7f728c056090 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7290ee8640 5 monclient: authenticate success, global_id 4652 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7289573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7289573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7289573640 10 monclient: dump: 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.293+0000 7f7290ee8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:45.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.295+0000 7f7289573640 1 -- 10.20.193.90:0/1177831247 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f727c003380 con 0x7f728c056090 2026-02-25T15:01:45.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.295+0000 7f7289573640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:45.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.295+0000 7f7289573640 1 -- 10.20.193.90:0/1177831247 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f727c0037c0 con 0x7f728c056090 2026-02-25T15:01:45.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.295+0000 7f7289573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.295+0000 7f7289573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:45.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.295+0000 7f7289573640 10 monclient: dump: 2026-02-25T15:01:45.296 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.295+0000 7f7290ee8640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.295+0000 7f7290ee8640 4 set_mon_vals no callback set 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.295+0000 7f7290ee8640 10 monclient: shutdown 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.295+0000 7f7290ee8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.295+0000 7f7290ee8640 1 -- 10.20.193.90:0/1177831247 >> v1:10.20.193.94:6789/0 conn(0x7f728c056090 legacy=0x7f728c0b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.295+0000 7f7290ee8640 1 -- 10.20.193.90:0/1177831247 shutdown_connections 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.295+0000 7f7290ee8640 1 -- 10.20.193.90:0/1177831247 >> 10.20.193.90:0/1177831247 conn(0x7f728c0b78c0 msgr2=0x7f728c0b7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.295+0000 7f7290ee8640 1 -- 10.20.193.90:0/1177831247 shutdown_connections 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.295+0000 7f7290ee8640 1 -- 10.20.193.90:0/1177831247 wait complete. 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.296+0000 7f7290ee8640 10 monclient: build_initial_monmap 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.296+0000 7f7290ee8640 10 monclient: monmap: 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:45.297 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.296+0000 7f7290ee8640 1 librados: starting msgr at 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.296+0000 7f7290ee8640 1 librados: starting objecter 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.296+0000 7f7290ee8640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.296+0000 7f7290ee8640 1 Processor -- start 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.296+0000 7f7290ee8640 1 -- start start 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.296+0000 7f7290ee8640 1 librados: setting wanted keys 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.296+0000 7f7290ee8640 1 librados: calling monclient init 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.296+0000 7f7290ee8640 10 monclient: init 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7290ee8640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7290ee8640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7290ee8640 10 monclient(hunting): picked mon.noname-b con 0x7f728c056090 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7290ee8640 10 monclient(hunting): picked mon.noname-c con 0x7f728c159da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7290ee8640 10 monclient(hunting): picked mon.noname-a con 0x7f728c165880 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7290ee8640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f728c152d20 con 0x7f728c165880 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7290ee8640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f728c1fb300 con 0x7f728c056090 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7290ee8640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f728c1fb5e0 con 0x7f728c159da0 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7290ee8640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7290ee8640 10 monclient(hunting): authenticate will time out at 2994.897256s 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7289d74640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f728c159da0 0x7f728c1364e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45958/0 (socket says 10.20.193.90:45958) 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f728ad76640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f728c165880 0x7f728c0b9bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57528/0 (socket says 10.20.193.90:57528) 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7289d74640 1 -- 10.20.193.90:0/2844899465 learned_addr learned my addr 10.20.193.90:0/2844899465 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:45.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7272ffd640 10 client.?.objecter ms_handle_connect 0x7f728c165880 2026-02-25T15:01:45.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7272ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7272ffd640 10 client.?.objecter ms_handle_connect 0x7f728c159da0 2026-02-25T15:01:45.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.297+0000 7f7272ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.298+0000 7f7272ffd640 1 -- 10.20.193.90:0/2844899465 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3782867154 0 0) 0x7f728c152d20 con 0x7f728c165880 2026-02-25T15:01:45.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.298+0000 7f7272ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.298+0000 7f7272ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.298+0000 7f7272ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.298+0000 7f7272ffd640 10 monclient(hunting): my global_id is 4647 2026-02-25T15:01:45.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.298+0000 7f7272ffd640 1 -- 10.20.193.90:0/2844899465 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7258003610 con 0x7f728c165880 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.301+0000 7f7272ffd640 1 -- 10.20.193.90:0/2844899465 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 658916498 0 0) 0x7f7258003610 con 0x7f728c165880 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.301+0000 7f7272ffd640 1 -- 10.20.193.90:0/2844899465 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f728c152d20 con 0x7f728c165880 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.301+0000 7f7272ffd640 1 -- 10.20.193.90:0/2844899465 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f727c0036d0 con 0x7f728c165880 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.301+0000 7f7272ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.301+0000 7f7272ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.301+0000 7f7272ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.301+0000 7f7272ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7272ffd640 1 -- 10.20.193.90:0/2844899465 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1498450756 0 0) 0x7f728c152d20 con 0x7f728c165880 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7272ffd640 1 -- 10.20.193.90:0/2844899465 >> v1:10.20.193.94:6789/0 conn(0x7f728c159da0 legacy=0x7f728c1364e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7272ffd640 1 -- 10.20.193.90:0/2844899465 >> v1:10.20.193.93:6789/0 conn(0x7f728c056090 legacy=0x7f728c152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7272ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7272ffd640 1 monclient: found mon.a 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7272ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7272ffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7272ffd640 1 -- 10.20.193.90:0/2844899465 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c1fb8c0 con 0x7f728c165880 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7272ffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7272ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7272ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7290ee8640 5 monclient: authenticate success, global_id 4647 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7290ee8640 10 monclient: _renew_subs 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7290ee8640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7290ee8640 1 -- 10.20.193.90:0/2844899465 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f728c1fbba0 con 0x7f728c165880 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7290ee8640 10 client.4647.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7290ee8640 10 monclient: _renew_subs 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7290ee8640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7290ee8640 1 -- 10.20.193.90:0/2844899465 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f728c136bf0 con 0x7f728c165880 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7290ee8640 1 librados: init done 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7290ee8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7290ee8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7290ee8640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.303+0000 7f7290ee8640 1 -- 10.20.193.90:0/2844899465 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f728c136e80 con 0x7f728c165880 2026-02-25T15:01:45.354 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.353+0000 7f9278b71640 10 monclient: get_monmap_and_config 2026-02-25T15:01:45.354 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.353+0000 7f9278b71640 10 monclient: build_initial_monmap 2026-02-25T15:01:45.354 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.353+0000 7f9278b71640 10 monclient: monmap: 2026-02-25T15:01:45.354 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:45.354 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.354 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:45.354290+0000 2026-02-25T15:01:45.354 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:45.354290+0000 2026-02-25T15:01:45.354 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:45.354 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.354 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:45.354 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:45.354 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:45.354 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.354+0000 7f9278b71640 1 Processor -- start 2026-02-25T15:01:45.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.354+0000 7f9278b71640 1 -- start start 2026-02-25T15:01:45.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.354+0000 7f9278b71640 10 monclient: init 2026-02-25T15:01:45.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.354+0000 7f9278b71640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:45.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.354+0000 7f9278b71640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:45.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.354+0000 7f9278b71640 10 monclient(hunting): picked mon.noname-c con 0x7f927415d0c0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:45.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.354+0000 7f9278b71640 10 monclient(hunting): picked mon.noname-a con 0x7f92740b0ca0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:45.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.354+0000 7f9278b71640 10 monclient(hunting): picked mon.noname-b con 0x7f92740b28b0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:45.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.354+0000 7f9278b71640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92740b8540 con 0x7f92740b0ca0 2026-02-25T15:01:45.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.354+0000 7f9278b71640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92740b5da0 con 0x7f92740b28b0 2026-02-25T15:01:45.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.354+0000 7f9278b71640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92740b6080 con 0x7f927415d0c0 2026-02-25T15:01:45.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.354+0000 7f9278b71640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:45.356 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.354+0000 7f9278b71640 10 monclient(hunting): authenticate will time out at 2994.954744s 2026-02-25T15:01:45.356 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.355+0000 7f9272575640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f927415d0c0 0x7f92740b0590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45970/0 (socket says 10.20.193.90:45970) 2026-02-25T15:01:45.356 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.355+0000 7f9272575640 1 -- 10.20.193.90:0/3207733767 learned_addr learned my addr 10.20.193.90:0/3207733767 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.356+0000 7fd75ab98640 10 monclient: get_monmap_and_config 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.356+0000 7fd75ab98640 10 monclient: build_initial_monmap 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.356+0000 7fd75ab98640 10 monclient: monmap: 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:45.356792+0000 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:45.356792+0000 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.356+0000 7fd75ab98640 1 Processor -- start 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.356+0000 7fd75ab98640 1 -- start start 2026-02-25T15:01:45.357 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.356+0000 7fd75ab98640 10 monclient: init 2026-02-25T15:01:45.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.357+0000 7fd75ab98640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:45.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.357+0000 7fd75ab98640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:45.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.357+0000 7fd75ab98640 10 monclient(hunting): picked mon.noname-b con 0x7fd75415d0c0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:45.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.357+0000 7fd75ab98640 10 monclient(hunting): picked mon.noname-a con 0x7fd7540b0ca0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:45.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.357+0000 7fd75ab98640 10 monclient(hunting): picked mon.noname-c con 0x7fd7540b28b0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:45.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.357+0000 7fd75ab98640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7540b8540 con 0x7fd7540b0ca0 2026-02-25T15:01:45.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.357+0000 7fd75ab98640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7540b5da0 con 0x7fd75415d0c0 2026-02-25T15:01:45.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.357+0000 7fd75ab98640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7540b6080 con 0x7fd7540b28b0 2026-02-25T15:01:45.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.357+0000 7fd75ab98640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:45.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.357+0000 7fd75ab98640 10 monclient(hunting): authenticate will time out at 2994.957129s 2026-02-25T15:01:45.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.357+0000 7fd75910e640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fd7540b28b0 0x7fd7540b7e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:45984/0 (socket says 10.20.193.90:45984) 2026-02-25T15:01:45.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.357+0000 7fd75910e640 1 -- 10.20.193.90:0/3210787882 learned_addr learned my addr 10.20.193.90:0/3210787882 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:45.359 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.358+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f385005f540 con 0x7f385c157780 2026-02-25T15:01:45.359 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.358+0000 7f383b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.359 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.358+0000 7f383b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:45.379 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.378+0000 7f4961ee1640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-25T15:01:45.379 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.378+0000 7f4961ee1640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-25T15:01:45.379 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.378+0000 7f4961ee1640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:45.379 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.378+0000 7f4961ee1640 1 -- 10.20.193.90:0/200253457 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f495c15bc20 con 0x7f495c166f60 2026-02-25T15:01:45.381 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.380+0000 7f4937fff640 1 -- 10.20.193.90:0/200253457 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (unknown 832126871 0 1257367751) 0x7f495005dd60 con 0x7f495c166f60 2026-02-25T15:01:45.381 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.380+0000 7f4937fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-25T15:01:45.381 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.380+0000 7f4937fff640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:45.381 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:45.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.383+0000 7f4961ee1640 10 librados: watch_flush enter 2026-02-25T15:01:45.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.383+0000 7f4961ee1640 10 librados: watch_flush exit 2026-02-25T15:01:45.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.383+0000 7f4961ee1640 20 client.4649.objecter shutdown clearing up homeless session... 2026-02-25T15:01:45.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.383+0000 7f4961ee1640 10 client.4649.objecter successfully canceled tick 2026-02-25T15:01:45.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.383+0000 7f4961ee1640 1 -- 10.20.193.90:0/200253457 >> v1:10.20.193.189:6800/3647375982 conn(0x7f492c07a0f0 legacy=0x7f492c09a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.383+0000 7f4961ee1640 10 monclient: shutdown 2026-02-25T15:01:45.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.383+0000 7f4961ee1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:45.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.383+0000 7f4961ee1640 1 -- 10.20.193.90:0/200253457 >> v1:10.20.193.94:6789/0 conn(0x7f495c166f60 legacy=0x7f495c13c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.383+0000 7f4961ee1640 1 -- 10.20.193.90:0/200253457 shutdown_connections 2026-02-25T15:01:45.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.383+0000 7f4961ee1640 1 -- 10.20.193.90:0/200253457 >> 10.20.193.90:0/200253457 conn(0x7f495c0b2ef0 msgr2=0x7f495c165080 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:45.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.383+0000 7f4961ee1640 1 -- 10.20.193.90:0/200253457 shutdown_connections 2026-02-25T15:01:45.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.383+0000 7f4961ee1640 1 -- 10.20.193.90:0/200253457 wait complete. 2026-02-25T15:01:45.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.383+0000 7f4961ee1640 1 librados: shutdown 2026-02-25T15:01:45.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.1 2026-02-25T15:01:45.396 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.401+0000 7f7272ffd640 1 -- 10.20.193.90:0/2844899465 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f727c002b10 con 0x7f728c165880 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.401+0000 7f7272ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.401+0000 7f7272ffd640 1 -- 10.20.193.90:0/2844899465 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f727c005200 con 0x7f728c165880 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.401+0000 7f7272ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.401+0000 7f7272ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.401+0000 7f7288d72640 4 set_mon_vals no callback set 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.401+0000 7f7272ffd640 10 monclient: dump: 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.402 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.402+0000 7f7272ffd640 1 -- 10.20.193.90:0/2844899465 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f727c01e6e0 con 0x7f728c165880 2026-02-25T15:01:45.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.402+0000 7f728a575640 10 monclient: get_auth_request con 0x7f7258079e90 auth_method 0 2026-02-25T15:01:45.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.403+0000 7f7272ffd640 10 client.4647.objecter ms_handle_connect 0x7f7258079e90 2026-02-25T15:01:45.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.405+0000 7f7272ffd640 1 -- 10.20.193.90:0/2844899465 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(28..28 src has 1..28) ==== 8169+0+0 (unknown 62831368 0 0) 0x7f727c0961a0 con 0x7f728c165880 2026-02-25T15:01:45.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.405+0000 7f7272ffd640 10 client.4647.objecter ms_dispatch2 0x7f728c000b90 osd_map(28..28 src has 1..28) 2026-02-25T15:01:45.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.405+0000 7f7272ffd640 3 client.4647.objecter handle_osd_map got epochs [28,28] > 0 2026-02-25T15:01:45.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.405+0000 7f7272ffd640 3 client.4647.objecter handle_osd_map decoding full epoch 28 2026-02-25T15:01:45.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.405+0000 7f7272ffd640 20 client.4647.objecter dump_active .. 0 homeless 2026-02-25T15:01:45.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.405+0000 7f7272ffd640 1 -- 10.20.193.90:0/2844899465 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f727c05fb00 con 0x7f728c165880 2026-02-25T15:01:45.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.405+0000 7f7272ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.405+0000 7f7272ffd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:45.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.411+0000 7f9271573640 1 -- 10.20.193.90:0/3207733767 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2836436007 0 0) 0x7f92740b8540 con 0x7f92740b0ca0 2026-02-25T15:01:45.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.411+0000 7f9271573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.411+0000 7f9271573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.411+0000 7f9271573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.411+0000 7f9271573640 10 monclient(hunting): my global_id is 4650 2026-02-25T15:01:45.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.411+0000 7fd74b7fe640 1 -- 10.20.193.90:0/3210787882 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2803055977 0 0) 0x7fd7540b8540 con 0x7fd7540b0ca0 2026-02-25T15:01:45.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.411+0000 7fd74b7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.411+0000 7fd74b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.411+0000 7fd74b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.411+0000 7fd74b7fe640 10 monclient(hunting): my global_id is 4653 2026-02-25T15:01:45.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.411+0000 7f9271573640 1 -- 10.20.193.90:0/3207733767 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9260003610 con 0x7f92740b0ca0 2026-02-25T15:01:45.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.411+0000 7fd74b7fe640 1 -- 10.20.193.90:0/3210787882 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd740003610 con 0x7fd7540b0ca0 2026-02-25T15:01:45.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7f9271573640 1 -- 10.20.193.90:0/3207733767 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 930574025 0 0) 0x7f92740b5da0 con 0x7f92740b28b0 2026-02-25T15:01:45.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7f9271573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7f9271573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7f9271573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7f9271573640 10 monclient(hunting): my global_id is 4627 2026-02-25T15:01:45.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7f9271573640 1 -- 10.20.193.90:0/3207733767 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92740b8540 con 0x7f92740b28b0 2026-02-25T15:01:45.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 1 -- 10.20.193.90:0/3210787882 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2609430080 0 0) 0x7fd7540b5da0 con 0x7fd75415d0c0 2026-02-25T15:01:45.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 10 monclient(hunting): my global_id is 4630 2026-02-25T15:01:45.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 1 -- 10.20.193.90:0/3210787882 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7540b8540 con 0x7fd75415d0c0 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 1 -- 10.20.193.90:0/3210787882 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3605063503 0 0) 0x7fd740003610 con 0x7fd7540b0ca0 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 1 -- 10.20.193.90:0/3210787882 >> v1:10.20.193.94:6789/0 conn(0x7fd7540b28b0 legacy=0x7fd7540b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 1 -- 10.20.193.90:0/3210787882 >> v1:10.20.193.93:6789/0 conn(0x7fd75415d0c0 legacy=0x7fd7540b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 1 monclient: found mon.noname-a 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 1 -- 10.20.193.90:0/3210787882 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd754163e10 con 0x7fd7540b0ca0 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 1 -- 10.20.193.90:0/3210787882 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd73c002df0 con 0x7fd7540b0ca0 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd75ab98640 5 monclient: authenticate success, global_id 4653 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd75ab98640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.450+0000 7fd74b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.451+0000 7fd74b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.451+0000 7fd74b7fe640 10 monclient: dump: 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.451+0000 7fd75ab98640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.451+0000 7f9271573640 1 -- 10.20.193.90:0/3207733767 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3958816515 0 0) 0x7f92740b6080 con 0x7f927415d0c0 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.451+0000 7f9271573640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.451+0000 7f9271573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.451+0000 7f9271573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.451+0000 7f9271573640 10 monclient(hunting): my global_id is 4655 2026-02-25T15:01:45.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.451+0000 7f9271573640 1 -- 10.20.193.90:0/3207733767 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92740b5da0 con 0x7f927415d0c0 2026-02-25T15:01:45.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9271573640 1 -- 10.20.193.90:0/3207733767 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2791392602 0 0) 0x7f9260003610 con 0x7f92740b0ca0 2026-02-25T15:01:45.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9271573640 1 -- 10.20.193.90:0/3207733767 >> v1:10.20.193.94:6789/0 conn(0x7f927415d0c0 legacy=0x7f92740b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9271573640 1 -- 10.20.193.90:0/3207733767 >> v1:10.20.193.93:6789/0 conn(0x7f92740b28b0 legacy=0x7f92740b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9271573640 10 monclient: _finish_hunting 0 2026-02-25T15:01:45.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9271573640 1 monclient: found mon.noname-a 2026-02-25T15:01:45.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9271573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:45.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9271573640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9271573640 1 -- 10.20.193.90:0/3207733767 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9274163e10 con 0x7f92740b0ca0 2026-02-25T15:01:45.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9271573640 10 monclient: _finish_auth 0 2026-02-25T15:01:45.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9271573640 10 monclient: _check_auth_tickets 2026-02-25T15:01:45.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9271573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:45.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9271573640 1 -- 10.20.193.90:0/3207733767 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f925c002df0 con 0x7f92740b0ca0 2026-02-25T15:01:45.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9271573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9271573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:45.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9271573640 10 monclient: dump: 2026-02-25T15:01:45.454 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.454 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.454 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.454 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.454 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.454 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.454 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.454 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.454 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.454 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9278b71640 5 monclient: authenticate success, global_id 4650 2026-02-25T15:01:45.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.452+0000 7f9278b71640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.458+0000 7f9271573640 1 -- 10.20.193.90:0/3207733767 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f925c002fc0 con 0x7f92740b0ca0 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.458+0000 7f9271573640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.458+0000 7f9271573640 1 -- 10.20.193.90:0/3207733767 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f925c003400 con 0x7f92740b0ca0 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.458+0000 7f9271573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.458+0000 7f9271573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.458+0000 7f9271573640 10 monclient: dump: 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.458+0000 7f9278b71640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.458+0000 7f9278b71640 4 set_mon_vals no callback set 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.458+0000 7f9278b71640 10 monclient: shutdown 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.458+0000 7f9278b71640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:45.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.458+0000 7f9278b71640 1 -- 10.20.193.90:0/3207733767 >> v1:10.20.193.90:6789/0 conn(0x7f92740b0ca0 legacy=0x7f92740b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.459+0000 7f9278b71640 1 -- 10.20.193.90:0/3207733767 shutdown_connections 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.459+0000 7f9278b71640 1 -- 10.20.193.90:0/3207733767 >> 10.20.193.90:0/3207733767 conn(0x7f92740b5290 msgr2=0x7f92740b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.459+0000 7f9278b71640 1 -- 10.20.193.90:0/3207733767 shutdown_connections 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.459+0000 7f9278b71640 1 -- 10.20.193.90:0/3207733767 wait complete. 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.459+0000 7f9278b71640 10 monclient: build_initial_monmap 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.459+0000 7f9278b71640 10 monclient: monmap: 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:45.460 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.459+0000 7f9278b71640 1 librados: starting msgr at 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.459+0000 7f9278b71640 1 librados: starting objecter 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.459+0000 7f9278b71640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.459+0000 7f9278b71640 1 Processor -- start 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.459+0000 7f9278b71640 1 -- start start 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.459+0000 7f9278b71640 1 librados: setting wanted keys 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.459+0000 7f9278b71640 1 librados: calling monclient init 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.459+0000 7f9278b71640 10 monclient: init 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.460+0000 7f9278b71640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.460+0000 7f9278b71640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.460+0000 7f9278b71640 10 monclient(hunting): picked mon.noname-b con 0x7f92740b0ca0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.460+0000 7f9278b71640 10 monclient(hunting): picked mon.noname-c con 0x7f92740b28b0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.460+0000 7f9278b71640 10 monclient(hunting): picked mon.noname-a con 0x7f927415d0c0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.460+0000 7f9278b71640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9274158e60 con 0x7f927415d0c0 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.460+0000 7f9278b71640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92741fc270 con 0x7f92740b0ca0 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.460+0000 7f9278b71640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9274134170 con 0x7f92740b28b0 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.460+0000 7f9278b71640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.460+0000 7f9278b71640 10 monclient(hunting): authenticate will time out at 2995.060528s 2026-02-25T15:01:45.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.460+0000 7f9272575640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f92740b0ca0 0x7f9274136700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:51298/0 (socket says 10.20.193.90:51298) 2026-02-25T15:01:45.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.460+0000 7f9272575640 1 -- 10.20.193.90:0/565017289 learned_addr learned my addr 10.20.193.90:0/565017289 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:45.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.461+0000 7f925affd640 10 client.?.objecter ms_handle_connect 0x7f92740b0ca0 2026-02-25T15:01:45.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.461+0000 7f925affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.461+0000 7fd74b7fe640 1 -- 10.20.193.90:0/3210787882 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd73c002fc0 con 0x7fd7540b0ca0 2026-02-25T15:01:45.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.461+0000 7fd74b7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:45.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.461+0000 7fd74b7fe640 1 -- 10.20.193.90:0/3210787882 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd73c003400 con 0x7fd7540b0ca0 2026-02-25T15:01:45.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.461+0000 7fd75ab98640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:45.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.461+0000 7fd75ab98640 4 set_mon_vals no callback set 2026-02-25T15:01:45.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.461+0000 7fd74b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.461+0000 7fd74b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:45.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.461+0000 7fd74b7fe640 10 monclient: dump: 2026-02-25T15:01:45.462 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.462 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.462 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.462 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.462+0000 7f925affd640 10 client.?.objecter ms_handle_connect 0x7f927415d0c0 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.462+0000 7f925affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.462+0000 7f925affd640 10 client.?.objecter ms_handle_connect 0x7f92740b28b0 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.462+0000 7f925affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.462+0000 7fd75ab98640 10 monclient: shutdown 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.462+0000 7fd75ab98640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.462+0000 7fd75ab98640 1 -- 10.20.193.90:0/3210787882 >> v1:10.20.193.90:6789/0 conn(0x7fd7540b0ca0 legacy=0x7fd7540b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.462+0000 7fd75ab98640 1 -- 10.20.193.90:0/3210787882 shutdown_connections 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.462+0000 7fd75ab98640 1 -- 10.20.193.90:0/3210787882 >> 10.20.193.90:0/3210787882 conn(0x7fd7540b5290 msgr2=0x7fd7540b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.462+0000 7fd75ab98640 1 -- 10.20.193.90:0/3210787882 shutdown_connections 2026-02-25T15:01:45.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.462+0000 7fd75ab98640 1 -- 10.20.193.90:0/3210787882 wait complete. 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 10 monclient: build_initial_monmap 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 10 monclient: monmap: 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 1 librados: starting msgr at 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 1 librados: starting objecter 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 1 Processor -- start 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 1 -- start start 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 1 librados: setting wanted keys 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 1 librados: calling monclient init 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 10 monclient: init 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:45.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 10 monclient(hunting): picked mon.noname-c con 0x7fd7540b0ca0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:45.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 10 monclient(hunting): picked mon.noname-a con 0x7fd7540b28b0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:45.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 10 monclient(hunting): picked mon.noname-b con 0x7fd75415d0c0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:45.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd75415c090 con 0x7fd7540b28b0 2026-02-25T15:01:45.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7541566d0 con 0x7fd75415d0c0 2026-02-25T15:01:45.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7541569b0 con 0x7fd7540b0ca0 2026-02-25T15:01:45.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:45.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.463+0000 7fd75ab98640 10 monclient(hunting): authenticate will time out at 2995.063972s 2026-02-25T15:01:45.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.464+0000 7fd75910e640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fd75415d0c0 0x7fd7541fb980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:51306/0 (socket says 10.20.193.90:51306) 2026-02-25T15:01:45.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.464+0000 7fd75910e640 1 -- 10.20.193.90:0/1034953126 learned_addr learned my addr 10.20.193.90:0/1034953126 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:45.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.464+0000 7fd7497fa640 10 client.?.objecter ms_handle_connect 0x7fd75415d0c0 2026-02-25T15:01:45.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.464+0000 7fd7497fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.465+0000 7f925affd640 1 -- 10.20.193.90:0/565017289 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1990731736 0 0) 0x7f9274158e60 con 0x7f927415d0c0 2026-02-25T15:01:45.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.465+0000 7f925affd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.465+0000 7f925affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.465+0000 7f925affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.465+0000 7f925affd640 10 monclient(hunting): my global_id is 4656 2026-02-25T15:01:45.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.465+0000 7f925affd640 1 -- 10.20.193.90:0/565017289 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9240003610 con 0x7f927415d0c0 2026-02-25T15:01:45.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.465+0000 7fd7497fa640 10 client.?.objecter ms_handle_connect 0x7fd7540b0ca0 2026-02-25T15:01:45.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.465+0000 7fd7497fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.465+0000 7fd7497fa640 10 client.?.objecter ms_handle_connect 0x7fd7540b28b0 2026-02-25T15:01:45.466 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.465+0000 7fd7497fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.471+0000 7f38615fa640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-25T15:01:45.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.471+0000 7f38615fa640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-25T15:01:45.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.471+0000 7f38615fa640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:01:45.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.471+0000 7f38615fa640 1 -- 10.20.193.90:0/1775404871 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f385c05e250 con 0x7f385c157780 2026-02-25T15:01:45.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.474+0000 7f383b7fe640 1 -- 10.20.193.90:0/1775404871 <== mon.1 v1:10.20.193.93:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (unknown 574334944 0 1257367751) 0x7f385c05e250 con 0x7f385c157780 2026-02-25T15:01:45.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.474+0000 7f383b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-25T15:01:45.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.474+0000 7f383b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:45.475 INFO:teuthology.orchestra.run.trial090.stdout:90194313220 2026-02-25T15:01:45.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.476+0000 7f38615fa640 10 librados: watch_flush enter 2026-02-25T15:01:45.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.476+0000 7f38615fa640 10 librados: watch_flush exit 2026-02-25T15:01:45.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.476+0000 7f38615fa640 20 client.4624.objecter shutdown clearing up homeless session... 2026-02-25T15:01:45.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.476+0000 7f38615fa640 10 client.4624.objecter successfully canceled tick 2026-02-25T15:01:45.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.476+0000 7f38615fa640 1 -- 10.20.193.90:0/1775404871 >> v1:10.20.193.189:6800/3647375982 conn(0x7f3828079ea0 legacy=0x7f382809a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.476+0000 7f38615fa640 10 monclient: shutdown 2026-02-25T15:01:45.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.476+0000 7f38615fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:45.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.476+0000 7f38615fa640 1 -- 10.20.193.90:0/1775404871 >> v1:10.20.193.93:6789/0 conn(0x7f385c157780 legacy=0x7f385c13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.476+0000 7f38615fa640 1 -- 10.20.193.90:0/1775404871 shutdown_connections 2026-02-25T15:01:45.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.476+0000 7f38615fa640 1 -- 10.20.193.90:0/1775404871 >> 10.20.193.90:0/1775404871 conn(0x7f385c0b2ef0 msgr2=0x7f385c0b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:45.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.476+0000 7f38615fa640 1 -- 10.20.193.90:0/1775404871 shutdown_connections 2026-02-25T15:01:45.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.476+0000 7f38615fa640 1 -- 10.20.193.90:0/1775404871 wait complete. 2026-02-25T15:01:45.477 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.476+0000 7f38615fa640 1 librados: shutdown 2026-02-25T15:01:45.486 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.0 2026-02-25T15:01:45.486 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.497+0000 7f925affd640 1 -- 10.20.193.90:0/565017289 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3065729624 0 0) 0x7f9240003610 con 0x7f927415d0c0 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.497+0000 7f925affd640 1 -- 10.20.193.90:0/565017289 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9274158e60 con 0x7f927415d0c0 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.497+0000 7f925affd640 1 -- 10.20.193.90:0/565017289 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f9264003430 con 0x7f927415d0c0 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.497+0000 7f925affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.497+0000 7f925affd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.497+0000 7f925affd640 10 monclient(hunting): dump: 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.497+0000 7f925affd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.497+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1248320993 0 0) 0x7fd75415c090 con 0x7fd7540b28b0 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.497+0000 7fd7497fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.497+0000 7fd7497fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.497+0000 7fd7497fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.497+0000 7fd7497fa640 10 monclient(hunting): my global_id is 4659 2026-02-25T15:01:45.498 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.497+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd724003610 con 0x7fd7540b28b0 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f925affd640 1 -- 10.20.193.90:0/565017289 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2213296170 0 0) 0x7f9274158e60 con 0x7f927415d0c0 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f925affd640 1 -- 10.20.193.90:0/565017289 >> v1:10.20.193.94:6789/0 conn(0x7f92740b28b0 legacy=0x7f9274158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f925affd640 1 -- 10.20.193.90:0/565017289 >> v1:10.20.193.93:6789/0 conn(0x7f92740b0ca0 legacy=0x7f9274136700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f925affd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f925affd640 1 monclient: found mon.a 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f925affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f925affd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f925affd640 1 -- 10.20.193.90:0/565017289 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9274134450 con 0x7f927415d0c0 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f925affd640 10 monclient: _finish_auth 0 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f925affd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f925affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f9278b71640 5 monclient: authenticate success, global_id 4656 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f9278b71640 10 monclient: _renew_subs 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f9278b71640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f9278b71640 1 -- 10.20.193.90:0/565017289 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9274134730 con 0x7f927415d0c0 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f9278b71640 10 client.4656.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:45.503 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f9278b71640 10 monclient: _renew_subs 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f9278b71640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f9278b71640 1 -- 10.20.193.90:0/565017289 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9274138960 con 0x7f927415d0c0 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7f9278b71640 1 librados: init done 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3636893367 0 0) 0x7fd724003610 con 0x7fd7540b28b0 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.502+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd75415c090 con 0x7fd7540b28b0 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.503+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd73c0033f0 con 0x7fd7540b28b0 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.503+0000 7fd7497fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.503+0000 7fd7497fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.503+0000 7fd7497fa640 10 monclient(hunting): dump: 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.503+0000 7fd7497fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.503+0000 7f9278b71640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.503+0000 7f9278b71640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.503+0000 7f9278b71640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.504 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.503+0000 7f9278b71640 1 -- 10.20.193.90:0/565017289 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f927405e210 con 0x7f927415d0c0 2026-02-25T15:01:45.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.520+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1139459546 0 0) 0x7fd7541569b0 con 0x7fd7540b0ca0 2026-02-25T15:01:45.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.520+0000 7fd7497fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.520+0000 7fd7497fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.520+0000 7fd7497fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.520+0000 7fd7497fa640 10 monclient(hunting): my global_id is 4661 2026-02-25T15:01:45.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.520+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd724003610 con 0x7fd7540b0ca0 2026-02-25T15:01:45.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.522+0000 7f7290ee8640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-25T15:01:45.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.523+0000 7f7290ee8640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-25T15:01:45.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.523+0000 7f7290ee8640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.523+0000 7f7290ee8640 1 -- 10.20.193.90:0/2844899465 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f728c0b2fc0 con 0x7f728c165880 2026-02-25T15:01:45.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.526+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3350074497 0 0) 0x7fd7541566d0 con 0x7fd75415d0c0 2026-02-25T15:01:45.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.526+0000 7fd7497fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.526+0000 7fd7497fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.526+0000 7fd7497fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.526+0000 7fd7497fa640 10 monclient(hunting): my global_id is 4636 2026-02-25T15:01:45.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.526+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7541569b0 con 0x7fd75415d0c0 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.600+0000 7f925affd640 1 -- 10.20.193.90:0/565017289 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9264003a80 con 0x7f927415d0c0 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.600+0000 7f925affd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.600+0000 7f925affd640 1 -- 10.20.193.90:0/565017289 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f9264003dd0 con 0x7f927415d0c0 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.600+0000 7f925affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.600+0000 7f925affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.600+0000 7f9270d72640 4 set_mon_vals no callback set 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.600+0000 7f925affd640 10 monclient: dump: 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.601 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.601+0000 7f925affd640 1 -- 10.20.193.90:0/565017289 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f926401d2b0 con 0x7f927415d0c0 2026-02-25T15:01:45.602 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.601+0000 7f9272575640 10 monclient: get_auth_request con 0x7f9240079e80 auth_method 0 2026-02-25T15:01:45.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.602+0000 7f925affd640 10 client.4656.objecter ms_handle_connect 0x7f9240079e80 2026-02-25T15:01:45.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.604+0000 7f925affd640 1 -- 10.20.193.90:0/565017289 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(29..29 src has 1..29) ==== 8141+0+0 (unknown 2043172091 0 0) 0x7f9264094e20 con 0x7f927415d0c0 2026-02-25T15:01:45.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.604+0000 7f925affd640 10 client.4656.objecter ms_dispatch2 0x7f9274000b90 osd_map(29..29 src has 1..29) 2026-02-25T15:01:45.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.604+0000 7f925affd640 3 client.4656.objecter handle_osd_map got epochs [29,29] > 0 2026-02-25T15:01:45.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.604+0000 7f925affd640 3 client.4656.objecter handle_osd_map decoding full epoch 29 2026-02-25T15:01:45.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.604+0000 7f925affd640 20 client.4656.objecter dump_active .. 0 homeless 2026-02-25T15:01:45.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.604+0000 7f925affd640 1 -- 10.20.193.90:0/565017289 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f926405e780 con 0x7f927415d0c0 2026-02-25T15:01:45.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.604+0000 7f925affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.604+0000 7f925affd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1858045334 0 0) 0x7fd75415c090 con 0x7fd7540b28b0 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 >> v1:10.20.193.94:6789/0 conn(0x7fd7540b0ca0 legacy=0x7fd7541ff590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 >> v1:10.20.193.93:6789/0 conn(0x7fd75415d0c0 legacy=0x7fd7541fb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd7497fa640 10 monclient: _finish_hunting 0 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd7497fa640 1 monclient: found mon.a 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd7497fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd7497fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd754156c90 con 0x7fd7540b28b0 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd7497fa640 10 monclient: _finish_auth 0 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd7497fa640 10 monclient: _check_auth_tickets 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd7497fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd75ab98640 5 monclient: authenticate success, global_id 4659 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd75ab98640 10 monclient: _renew_subs 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd75ab98640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd75ab98640 1 -- 10.20.193.90:0/1034953126 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd7541fed90 con 0x7fd7540b28b0 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd75ab98640 10 client.4659.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd75ab98640 10 monclient: _renew_subs 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd75ab98640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd75ab98640 1 -- 10.20.193.90:0/1034953126 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd754134c10 con 0x7fd7540b28b0 2026-02-25T15:01:45.609 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.608+0000 7fd75ab98640 1 librados: init done 2026-02-25T15:01:45.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.609+0000 7fd75ab98640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.609+0000 7fd75ab98640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.609+0000 7fd75ab98640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.609+0000 7fd75ab98640 1 -- 10.20.193.90:0/1034953126 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd754156ed0 con 0x7fd7540b28b0 2026-02-25T15:01:45.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.609+0000 7f7272ffd640 1 -- 10.20.193.90:0/2844899465 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+12 (unknown 3207554136 0 1498640560) 0x7f727c05f400 con 0x7f728c165880 2026-02-25T15:01:45.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.610+0000 7f7272ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-25T15:01:45.610 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.610+0000 7f7272ffd640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:45.611 INFO:teuthology.orchestra.run.trial090.stdout:90194313221 2026-02-25T15:01:45.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.612+0000 7f7290ee8640 10 librados: watch_flush enter 2026-02-25T15:01:45.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.612+0000 7f7290ee8640 10 librados: watch_flush exit 2026-02-25T15:01:45.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.612+0000 7f7290ee8640 20 client.4647.objecter shutdown clearing up homeless session... 2026-02-25T15:01:45.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.612+0000 7f7290ee8640 10 client.4647.objecter successfully canceled tick 2026-02-25T15:01:45.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.612+0000 7f7290ee8640 1 -- 10.20.193.90:0/2844899465 >> v1:10.20.193.189:6800/3647375982 conn(0x7f7258079e90 legacy=0x7f725809a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.612+0000 7f7290ee8640 10 monclient: shutdown 2026-02-25T15:01:45.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.612+0000 7f7290ee8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:45.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.612+0000 7f7290ee8640 1 -- 10.20.193.90:0/2844899465 >> v1:10.20.193.90:6789/0 conn(0x7f728c165880 legacy=0x7f728c0b9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.612+0000 7f7290ee8640 1 -- 10.20.193.90:0/2844899465 shutdown_connections 2026-02-25T15:01:45.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.612+0000 7f7290ee8640 1 -- 10.20.193.90:0/2844899465 >> 10.20.193.90:0/2844899465 conn(0x7f728c0b78c0 msgr2=0x7f728c0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:45.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.612+0000 7f7290ee8640 1 -- 10.20.193.90:0/2844899465 shutdown_connections 2026-02-25T15:01:45.613 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.612+0000 7f7290ee8640 1 -- 10.20.193.90:0/2844899465 wait complete. 2026-02-25T15:01:45.614 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.612+0000 7f7290ee8640 1 librados: shutdown 2026-02-25T15:01:45.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313221 for osd.8 2026-02-25T15:01:45.623 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:45.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.716+0000 7f9278b71640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-25T15:01:45.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.716+0000 7f9278b71640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-25T15:01:45.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.716+0000 7f9278b71640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.716+0000 7f9278b71640 1 -- 10.20.193.90:0/565017289 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f92740b4c40 con 0x7f927415d0c0 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.743+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd73c002f70 con 0x7fd7540b28b0 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.743+0000 7fd7497fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.743+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd73c003b00 con 0x7fd7540b28b0 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.743+0000 7fd7497fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.743+0000 7fd7497fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.743+0000 7fd74affd640 4 set_mon_vals no callback set 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.743+0000 7fd7497fa640 10 monclient: dump: 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.743+0000 7f925affd640 1 -- 10.20.193.90:0/565017289 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+12 (unknown 2894986287 0 1498640560) 0x7f926405e080 con 0x7f927415d0c0 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.743+0000 7f925affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-25T15:01:45.744 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.743+0000 7f925affd640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:45.745 INFO:teuthology.orchestra.run.trial090.stdout:90194313221 2026-02-25T15:01:45.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.744+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fd73c01cfe0 con 0x7fd7540b28b0 2026-02-25T15:01:45.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.744+0000 7fd75890d640 10 monclient: get_auth_request con 0x7fd724079f90 auth_method 0 2026-02-25T15:01:45.745 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.744+0000 7fd7497fa640 10 client.4659.objecter ms_handle_connect 0x7fd724079f90 2026-02-25T15:01:45.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.745+0000 7f9278b71640 10 librados: watch_flush enter 2026-02-25T15:01:45.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.745+0000 7f9278b71640 10 librados: watch_flush exit 2026-02-25T15:01:45.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.745+0000 7f9278b71640 20 client.4656.objecter shutdown clearing up homeless session... 2026-02-25T15:01:45.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.745+0000 7f9278b71640 10 client.4656.objecter successfully canceled tick 2026-02-25T15:01:45.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.745+0000 7f9278b71640 1 -- 10.20.193.90:0/565017289 >> v1:10.20.193.189:6800/3647375982 conn(0x7f9240079e80 legacy=0x7f924009a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.745+0000 7f9278b71640 10 monclient: shutdown 2026-02-25T15:01:45.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.745+0000 7f9278b71640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:45.746 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.745+0000 7f9278b71640 1 -- 10.20.193.90:0/565017289 >> v1:10.20.193.90:6789/0 conn(0x7f927415d0c0 legacy=0x7f9274138250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.747 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.745+0000 7f9278b71640 1 -- 10.20.193.90:0/565017289 shutdown_connections 2026-02-25T15:01:45.747 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.745+0000 7f9278b71640 1 -- 10.20.193.90:0/565017289 >> 10.20.193.90:0/565017289 conn(0x7f92740b5290 msgr2=0x7f92740b6960 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:45.747 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.745+0000 7f9278b71640 1 -- 10.20.193.90:0/565017289 shutdown_connections 2026-02-25T15:01:45.747 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.745+0000 7f9278b71640 1 -- 10.20.193.90:0/565017289 wait complete. 2026-02-25T15:01:45.747 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.745+0000 7f9278b71640 1 librados: shutdown 2026-02-25T15:01:45.748 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.747+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(29..29 src has 1..29) ==== 8141+0+0 (unknown 2043172091 0 0) 0x7fd73c094a70 con 0x7fd7540b28b0 2026-02-25T15:01:45.748 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.747+0000 7fd7497fa640 10 client.4659.objecter ms_dispatch2 0x7fd754000b90 osd_map(29..29 src has 1..29) 2026-02-25T15:01:45.748 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.747+0000 7fd7497fa640 3 client.4659.objecter handle_osd_map got epochs [29,29] > 0 2026-02-25T15:01:45.748 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.747+0000 7fd7497fa640 3 client.4659.objecter handle_osd_map decoding full epoch 29 2026-02-25T15:01:45.748 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.747+0000 7fd7497fa640 20 client.4659.objecter dump_active .. 0 homeless 2026-02-25T15:01:45.748 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.747+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fd754156ed0 con 0x7fd7540b28b0 2026-02-25T15:01:45.748 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.747+0000 7fd7497fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.748 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.747+0000 7fd7497fa640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:45.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313221 for osd.9 2026-02-25T15:01:45.756 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:45.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.854+0000 7fd75ab98640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-25T15:01:45.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.854+0000 7fd75ab98640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-25T15:01:45.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.854+0000 7fd75ab98640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:45.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.854+0000 7fd75ab98640 1 -- 10.20.193.90:0/1034953126 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fd7540b1130 con 0x7fd7540b28b0 2026-02-25T15:01:45.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.857+0000 7fd7497fa640 1 -- 10.20.193.90:0/1034953126 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+12 (unknown 3646472635 0 1498640560) 0x7fd73c05e3d0 con 0x7fd7540b28b0 2026-02-25T15:01:45.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.857+0000 7fd7497fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-25T15:01:45.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.857+0000 7fd7497fa640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:45.858 INFO:teuthology.orchestra.run.trial090.stdout:90194313221 2026-02-25T15:01:45.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.858+0000 7fd75ab98640 10 librados: watch_flush enter 2026-02-25T15:01:45.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.858+0000 7fd75ab98640 10 librados: watch_flush exit 2026-02-25T15:01:45.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.859+0000 7fd75ab98640 20 client.4659.objecter shutdown clearing up homeless session... 2026-02-25T15:01:45.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.859+0000 7fd75ab98640 10 client.4659.objecter successfully canceled tick 2026-02-25T15:01:45.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.859+0000 7fd75ab98640 1 -- 10.20.193.90:0/1034953126 >> v1:10.20.193.189:6800/3647375982 conn(0x7fd724079f90 legacy=0x7fd72409a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.859+0000 7fd75ab98640 10 monclient: shutdown 2026-02-25T15:01:45.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.859+0000 7fd75ab98640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:45.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.859+0000 7fd75ab98640 1 -- 10.20.193.90:0/1034953126 >> v1:10.20.193.90:6789/0 conn(0x7fd7540b28b0 legacy=0x7fd75415b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.859+0000 7fd75ab98640 1 -- 10.20.193.90:0/1034953126 shutdown_connections 2026-02-25T15:01:45.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.859+0000 7fd75ab98640 1 -- 10.20.193.90:0/1034953126 >> 10.20.193.90:0/1034953126 conn(0x7fd7540b5290 msgr2=0x7fd7541633c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:45.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.859+0000 7fd75ab98640 1 -- 10.20.193.90:0/1034953126 shutdown_connections 2026-02-25T15:01:45.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.859+0000 7fd75ab98640 1 -- 10.20.193.90:0/1034953126 wait complete. 2026-02-25T15:01:45.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.859+0000 7fd75ab98640 1 librados: shutdown 2026-02-25T15:01:45.868 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313221 for osd.10 2026-02-25T15:01:45.868 DEBUG:teuthology.parallel:result is None 2026-02-25T15:01:45.869 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-25T15:01:45.869 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T15:01:45.937 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.936+0000 7f46a82e9640 10 monclient: get_monmap_and_config 2026-02-25T15:01:45.937 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.936+0000 7f46a82e9640 10 monclient: build_initial_monmap 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.936+0000 7f46a82e9640 10 monclient: monmap: 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:45.937500+0000 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:45.937500+0000 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.937+0000 7f46a82e9640 1 Processor -- start 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.937+0000 7f46a82e9640 1 -- start start 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.937+0000 7f46a82e9640 10 monclient: init 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.937+0000 7f46a82e9640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:45.938 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.937+0000 7f46a82e9640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:01:45.939 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.937+0000 7f46a82e9640 10 monclient(hunting): picked mon.noname-c con 0x7f46a0058da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:45.939 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.937+0000 7f46a82e9640 10 monclient(hunting): picked mon.noname-b con 0x7f46a00b0410 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:45.939 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.937+0000 7f46a82e9640 10 monclient(hunting): picked mon.noname-a con 0x7f46a01626a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:45.939 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.937+0000 7f46a82e9640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46a01655c0 con 0x7f46a01626a0 2026-02-25T15:01:45.939 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.937+0000 7f46a82e9640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46a00b12b0 con 0x7f46a00b0410 2026-02-25T15:01:45.939 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.937+0000 7f46a82e9640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46a00b1590 con 0x7f46a0058da0 2026-02-25T15:01:45.939 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.937+0000 7f46a82e9640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:45.939 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.937+0000 7f46a82e9640 10 monclient(hunting): authenticate will time out at 2995.537870s 2026-02-25T15:01:45.939 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.938+0000 7f46a585d640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f46a00b0410 0x7f46a0164eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:51308/0 (socket says 10.20.193.90:51308) 2026-02-25T15:01:45.939 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.938+0000 7f46a585d640 1 -- 10.20.193.90:0/962977804 learned_addr learned my addr 10.20.193.90:0/962977804 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:45.940 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 1 -- 10.20.193.90:0/962977804 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2741558745 0 0) 0x7f46a01655c0 con 0x7f46a01626a0 2026-02-25T15:01:45.940 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.940 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.940 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.940 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 10 monclient(hunting): my global_id is 4662 2026-02-25T15:01:45.940 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 1 -- 10.20.193.90:0/962977804 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4688003610 con 0x7f46a01626a0 2026-02-25T15:01:45.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 1 -- 10.20.193.90:0/962977804 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1896280058 0 0) 0x7f46a00b1590 con 0x7f46a0058da0 2026-02-25T15:01:45.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 10 monclient(hunting): my global_id is 4664 2026-02-25T15:01:45.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 1 -- 10.20.193.90:0/962977804 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46a01655c0 con 0x7f46a0058da0 2026-02-25T15:01:45.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 1 -- 10.20.193.90:0/962977804 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1785935611 0 0) 0x7f46a00b12b0 con 0x7f46a00b0410 2026-02-25T15:01:45.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 10 monclient(hunting): my global_id is 4639 2026-02-25T15:01:45.941 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.939+0000 7f46a505c640 1 -- 10.20.193.90:0/962977804 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46a00b1590 con 0x7f46a00b0410 2026-02-25T15:01:45.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a505c640 1 -- 10.20.193.90:0/962977804 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4248306534 0 0) 0x7f46a01655c0 con 0x7f46a0058da0 2026-02-25T15:01:45.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a505c640 1 -- 10.20.193.90:0/962977804 >> v1:10.20.193.93:6789/0 conn(0x7f46a00b0410 legacy=0x7f46a0164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a505c640 1 -- 10.20.193.90:0/962977804 >> v1:10.20.193.90:6789/0 conn(0x7f46a01626a0 legacy=0x7f46a01c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a505c640 10 monclient: _finish_hunting 0 2026-02-25T15:01:45.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a505c640 1 monclient: found mon.noname-c 2026-02-25T15:01:45.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a505c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:45.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a505c640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:45.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a505c640 1 -- 10.20.193.90:0/962977804 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a00b1870 con 0x7f46a0058da0 2026-02-25T15:01:45.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a505c640 10 monclient: _finish_auth 0 2026-02-25T15:01:45.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a505c640 10 monclient: _check_auth_tickets 2026-02-25T15:01:45.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a505c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:45.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a505c640 1 -- 10.20.193.90:0/962977804 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4690002df0 con 0x7f46a0058da0 2026-02-25T15:01:45.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a505c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.942 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a505c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:45.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a505c640 10 monclient: dump: 2026-02-25T15:01:45.943 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.943 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.943 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.943 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.943 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.943 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.943 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.943 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.943 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.943 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a82e9640 5 monclient: authenticate success, global_id 4664 2026-02-25T15:01:45.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.941+0000 7f46a82e9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:45.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.942+0000 7f46a505c640 1 -- 10.20.193.90:0/962977804 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4690002fc0 con 0x7f46a0058da0 2026-02-25T15:01:45.943 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.942+0000 7f46a505c640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.942+0000 7f46a505c640 1 -- 10.20.193.90:0/962977804 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4690003400 con 0x7f46a0058da0 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.942+0000 7f46a505c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.942+0000 7f46a505c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.942+0000 7f46a505c640 10 monclient: dump: 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.942+0000 7f46a82e9640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.942+0000 7f46a82e9640 4 set_mon_vals no callback set 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.942+0000 7f46a82e9640 10 monclient: shutdown 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.942+0000 7f46a82e9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.942+0000 7f46a82e9640 1 -- 10.20.193.90:0/962977804 >> v1:10.20.193.94:6789/0 conn(0x7f46a0058da0 legacy=0x7f46a01647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.942+0000 7f46a82e9640 1 -- 10.20.193.90:0/962977804 shutdown_connections 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.942+0000 7f46a82e9640 1 -- 10.20.193.90:0/962977804 >> 10.20.193.90:0/962977804 conn(0x7f46a00b65f0 msgr2=0x7f46a00b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.942+0000 7f46a82e9640 1 -- 10.20.193.90:0/962977804 shutdown_connections 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.942+0000 7f46a82e9640 1 -- 10.20.193.90:0/962977804 wait complete. 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 10 monclient: build_initial_monmap 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 10 monclient: monmap: 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:45.944 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:45.945 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:45.945 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:45.945 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:45.945 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 1 librados: starting msgr at 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 1 librados: starting objecter 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 1 Processor -- start 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 1 -- start start 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 1 librados: setting wanted keys 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 1 librados: calling monclient init 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 10 monclient: init 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 10 monclient(hunting): picked mon.noname-a con 0x7f46a0058da0 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 10 monclient(hunting): picked mon.noname-c con 0x7f46a00b0410 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 10 monclient(hunting): picked mon.noname-b con 0x7f46a01626a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46a0165a60 con 0x7f46a0058da0 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46a0131a60 con 0x7f46a01626a0 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46a0131d40 con 0x7f46a00b0410 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a82e9640 10 monclient(hunting): authenticate will time out at 2995.543717s 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a605e640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f46a0058da0 0x7f46a0165350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57566/0 (socket says 10.20.193.90:57566) 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.943+0000 7f46a605e640 1 -- 10.20.193.90:0/220074629 learned_addr learned my addr 10.20.193.90:0/220074629 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.944+0000 7f4686ffd640 10 client.?.objecter ms_handle_connect 0x7f46a0058da0 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.944+0000 7f4686ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.944+0000 7f4686ffd640 10 client.?.objecter ms_handle_connect 0x7f46a00b0410 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.944+0000 7f4686ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.944+0000 7f4686ffd640 10 client.?.objecter ms_handle_connect 0x7f46a01626a0 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.944+0000 7f4686ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1457291332 0 0) 0x7f46a0165a60 con 0x7f46a0058da0 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 10 monclient(hunting): my global_id is 4665 2026-02-25T15:01:45.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4670003610 con 0x7f46a0058da0 2026-02-25T15:01:45.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 622317492 0 0) 0x7f46a0131d40 con 0x7f46a00b0410 2026-02-25T15:01:45.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 10 monclient(hunting): my global_id is 4667 2026-02-25T15:01:45.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46a0165a60 con 0x7f46a00b0410 2026-02-25T15:01:45.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2415842193 0 0) 0x7f46a0131a60 con 0x7f46a01626a0 2026-02-25T15:01:45.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:45.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:45.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:45.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 10 monclient(hunting): my global_id is 4642 2026-02-25T15:01:45.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.945+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46a0131d40 con 0x7f46a01626a0 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1984996910 0 0) 0x7f46a0165a60 con 0x7f46a00b0410 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f46a0131a60 con 0x7f46a00b0410 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4694003060 con 0x7f46a00b0410 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.948 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2529565973 0 0) 0x7f4670003610 con 0x7f46a0058da0 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f46a0165a60 con 0x7f46a0058da0 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f46900033d0 con 0x7f46a0058da0 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1645816925 0 0) 0x7f46a0131d40 con 0x7f46a01626a0 2026-02-25T15:01:45.949 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4670003610 con 0x7f46a01626a0 2026-02-25T15:01:45.950 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f469c003250 con 0x7f46a01626a0 2026-02-25T15:01:45.950 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:45.950 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:45.950 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.947+0000 7f4686ffd640 10 monclient(hunting): dump: 2026-02-25T15:01:45.950 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:45.950 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:45.950 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.950 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:45.950 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:45.950 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:45.950 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:45.950 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:45.950 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:45.950 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3328419175 0 0) 0x7f46a0131a60 con 0x7f46a00b0410 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 >> v1:10.20.193.93:6789/0 conn(0x7f46a01626a0 legacy=0x7f46a01309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 >> v1:10.20.193.90:6789/0 conn(0x7f46a0058da0 legacy=0x7f46a0165350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f4686ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f4686ffd640 1 monclient: found mon.c 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f4686ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f4686ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a0132020 con 0x7f46a00b0410 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f4686ffd640 10 monclient: _finish_auth 0 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f4686ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f4686ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f46a82e9640 5 monclient: authenticate success, global_id 4667 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f46a82e9640 10 monclient: _renew_subs 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f46a82e9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f46a82e9640 1 -- 10.20.193.90:0/220074629 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f46a0132300 con 0x7f46a00b0410 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f46a82e9640 10 client.4667.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f46a82e9640 10 monclient: _renew_subs 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f46a82e9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f46a82e9640 1 -- 10.20.193.90:0/220074629 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f46a00ae9e0 con 0x7f46a00b0410 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f46a82e9640 1 librados: init done 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f46a82e9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f46a82e9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f46a82e9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:45.951 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:45.949+0000 7f46a82e9640 1 -- 10.20.193.90:0/220074629 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a0164070 con 0x7f46a00b0410 2026-02-25T15:01:46.040 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.040+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f46940048d0 con 0x7f46a00b0410 2026-02-25T15:01:46.040 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.040+0000 7f4686ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.040+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4694004c20 con 0x7f46a00b0410 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.040+0000 7f4686ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.040+0000 7f4686ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.040+0000 7f4686ffd640 10 monclient: dump: 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.040+0000 7f46a485b640 4 set_mon_vals no callback set 2026-02-25T15:01:46.041 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.040+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f469401e100 con 0x7f46a00b0410 2026-02-25T15:01:46.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.041+0000 7f46a605e640 10 monclient: get_auth_request con 0x7f467007a040 auth_method 0 2026-02-25T15:01:46.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.041+0000 7f4686ffd640 10 client.4667.objecter ms_handle_connect 0x7f467007a040 2026-02-25T15:01:46.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.041+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(29..29 src has 1..29) ==== 8141+0+0 (unknown 2043172091 0 0) 0x7f4694095aa0 con 0x7f46a00b0410 2026-02-25T15:01:46.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.041+0000 7f4686ffd640 10 client.4667.objecter ms_dispatch2 0x7f46a0000b90 osd_map(29..29 src has 1..29) 2026-02-25T15:01:46.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.041+0000 7f4686ffd640 3 client.4667.objecter handle_osd_map got epochs [29,29] > 0 2026-02-25T15:01:46.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.041+0000 7f4686ffd640 3 client.4667.objecter handle_osd_map decoding full epoch 29 2026-02-25T15:01:46.042 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.041+0000 7f4686ffd640 20 client.4667.objecter dump_active .. 0 homeless 2026-02-25T15:01:46.044 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.043+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f46a0164070 con 0x7f46a00b0410 2026-02-25T15:01:46.044 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.043+0000 7f4686ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:46.044 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.043+0000 7f4686ffd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:46.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.149+0000 7f46a82e9640 1 -- 10.20.193.90:0/220074629 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46a00b26f0 con 0x7f467007a040 2026-02-25T15:01:46.151 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.150+0000 7f4686ffd640 1 -- 10.20.193.90:0/220074629 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+42839 (unknown 2965378022 0 333748029) 0x7f46a00b26f0 con 0x7f467007a040 2026-02-25T15:01:46.151 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:01:46.151 INFO:teuthology.orchestra.run.trial090.stderr:dumped all 2026-02-25T15:01:46.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.151+0000 7f46a82e9640 10 librados: watch_flush enter 2026-02-25T15:01:46.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.151+0000 7f46a82e9640 10 librados: watch_flush exit 2026-02-25T15:01:46.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.151+0000 7f46a82e9640 20 client.4667.objecter shutdown clearing up homeless session... 2026-02-25T15:01:46.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.151+0000 7f46a82e9640 10 client.4667.objecter successfully canceled tick 2026-02-25T15:01:46.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.151+0000 7f46a82e9640 1 -- 10.20.193.90:0/220074629 >> v1:10.20.193.189:6800/3647375982 conn(0x7f467007a040 legacy=0x7f467009a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.151+0000 7f46a82e9640 10 monclient: shutdown 2026-02-25T15:01:46.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.151+0000 7f46a82e9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:46.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.151+0000 7f46a82e9640 1 -- 10.20.193.90:0/220074629 >> v1:10.20.193.94:6789/0 conn(0x7f46a00b0410 legacy=0x7f46a00ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.151+0000 7f46a82e9640 1 -- 10.20.193.90:0/220074629 shutdown_connections 2026-02-25T15:01:46.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.151+0000 7f46a82e9640 1 -- 10.20.193.90:0/220074629 >> 10.20.193.90:0/220074629 conn(0x7f46a00b65f0 msgr2=0x7f46a0059bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:46.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.151+0000 7f46a82e9640 1 -- 10.20.193.90:0/220074629 shutdown_connections 2026-02-25T15:01:46.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.151+0000 7f46a82e9640 1 -- 10.20.193.90:0/220074629 wait complete. 2026-02-25T15:01:46.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.151+0000 7f46a82e9640 1 librados: shutdown 2026-02-25T15:01:46.161 INFO:teuthology.orchestra.run.trial090.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2026-02-25T15:01:45.418776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":436624,"kb_used_data":5264,"kb_used_omap":132,"kb_used_meta":430971,"kb_avail":1509512816,"statfs":{"total":1546188226560,"available":1545741123584,"internally_reserved":0,"allocated":5390336,"data_stored":2837206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":135852,"internal_metadata":441314644},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131107,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":18,"num_write_kb":264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.353284"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:40.614364+0000","last_change":"2026-02-25T15:01:40.617020+0000","last_active":"2026-02-25T15:01:40.614364+0000","last_peered":"2026-02-25T15:01:40.614364+0000","last_clean":"2026-02-25T15:01:40.614364+0000","last_became_active":"2026-02-25T15:01:38.997857+0000","last_became_peered":"2026-02-25T15:01:38.997857+0000","last_unstale":"2026-02-25T15:01:40.614364+0000","last_undegraded":"2026-02-25T15:01:40.614364+0000","last_fullsized":"2026-02-25T15:01:40.614364+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:49:56.506302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062975700999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14],"acting":[11,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:41.108756+0000","last_change":"2026-02-25T15:01:41.128386+0000","last_active":"2026-02-25T15:01:41.108756+0000","last_peered":"2026-02-25T15:01:41.108756+0000","last_clean":"2026-02-25T15:01:41.108756+0000","last_became_active":"2026-02-25T15:01:38.782341+0000","last_became_peered":"2026-02-25T15:01:38.782341+0000","last_unstale":"2026-02-25T15:01:41.108756+0000","last_undegraded":"2026-02-25T15:01:41.108756+0000","last_fullsized":"2026-02-25T15:01:41.108756+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:07:32.765627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031008292,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,3],"acting":[12,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:41.135348+0000","last_change":"2026-02-25T15:01:41.143783+0000","last_active":"2026-02-25T15:01:41.135348+0000","last_peered":"2026-02-25T15:01:41.135348+0000","last_clean":"2026-02-25T15:01:41.135348+0000","last_became_active":"2026-02-25T15:01:38.731569+0000","last_became_peered":"2026-02-25T15:01:38.731569+0000","last_unstale":"2026-02-25T15:01:41.135348+0000","last_undegraded":"2026-02-25T15:01:41.135348+0000","last_fullsized":"2026-02-25T15:01:41.135348+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:00:25.099174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037093318,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15],"acting":[7,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:41.116719+0000","last_change":"2026-02-25T15:01:41.139794+0000","last_active":"2026-02-25T15:01:41.116719+0000","last_peered":"2026-02-25T15:01:41.116719+0000","last_clean":"2026-02-25T15:01:41.116719+0000","last_became_active":"2026-02-25T15:01:38.780556+0000","last_became_peered":"2026-02-25T15:01:38.780556+0000","last_unstale":"2026-02-25T15:01:41.116719+0000","last_undegraded":"2026-02-25T15:01:41.116719+0000","last_fullsized":"2026-02-25T15:01:41.116719+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:57:49.842137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041710525999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:40.608781+0000","last_change":"2026-02-25T15:01:40.608781+0000","last_active":"2026-02-25T15:01:40.608781+0000","last_peered":"2026-02-25T15:01:40.608781+0000","last_clean":"2026-02-25T15:01:40.608781+0000","last_became_active":"2026-02-25T15:01:38.743941+0000","last_became_peered":"2026-02-25T15:01:38.743941+0000","last_unstale":"2026-02-25T15:01:40.608781+0000","last_undegraded":"2026-02-25T15:01:40.608781+0000","last_fullsized":"2026-02-25T15:01:40.608781+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:53:32.920667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.061484652000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13],"acting":[11,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:41.108353+0000","last_change":"2026-02-25T15:01:41.129594+0000","last_active":"2026-02-25T15:01:41.108353+0000","last_peered":"2026-02-25T15:01:41.108353+0000","last_clean":"2026-02-25T15:01:41.108353+0000","last_became_active":"2026-02-25T15:01:38.861789+0000","last_became_peered":"2026-02-25T15:01:38.861789+0000","last_unstale":"2026-02-25T15:01:41.108353+0000","last_undegraded":"2026-02-25T15:01:41.108353+0000","last_fullsized":"2026-02-25T15:01:41.108353+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:36:46.424677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032448785000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:41.097316+0000","last_change":"2026-02-25T15:01:41.116971+0000","last_active":"2026-02-25T15:01:41.097316+0000","last_peered":"2026-02-25T15:01:41.097316+0000","last_clean":"2026-02-25T15:01:41.097316+0000","last_became_active":"2026-02-25T15:01:38.737942+0000","last_became_peered":"2026-02-25T15:01:38.737942+0000","last_unstale":"2026-02-25T15:01:41.097316+0000","last_undegraded":"2026-02-25T15:01:41.097316+0000","last_fullsized":"2026-02-25T15:01:41.097316+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:51:33.138764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031295004000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13],"acting":[0,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:40.611559+0000","last_change":"2026-02-25T15:01:40.614625+0000","last_active":"2026-02-25T15:01:40.611559+0000","last_peered":"2026-02-25T15:01:40.611559+0000","last_clean":"2026-02-25T15:01:40.611559+0000","last_became_active":"2026-02-25T15:01:38.791923+0000","last_became_peered":"2026-02-25T15:01:38.791923+0000","last_unstale":"2026-02-25T15:01:40.611559+0000","last_undegraded":"2026-02-25T15:01:40.611559+0000","last_fullsized":"2026-02-25T15:01:40.611559+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:51:10.001377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062725172999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5],"acting":[11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"24'32","reported_seq":65,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:41.251864+0000","last_change":"2026-02-25T15:01:30.868615+0000","last_active":"2026-02-25T15:01:41.251864+0000","last_peered":"2026-02-25T15:01:41.251864+0000","last_clean":"2026-02-25T15:01:41.251864+0000","last_became_active":"2026-02-25T15:01:30.851092+0000","last_became_peered":"2026-02-25T15:01:30.851092+0000","last_unstale":"2026-02-25T15:01:41.251864+0000","last_undegraded":"2026-02-25T15:01:41.251864+0000","last_fullsized":"2026-02-25T15:01:41.251864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:29.405841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:29.405841+0000","last_clean_scrub_stamp":"2026-02-25T15:01:29.405841+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T02:31:46.315230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3],"acting":[8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":276,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":282624,"data_stored":119932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[0,2,3,4,6,7,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8120,"internal_metadata":27582536},"hb_peers":[0,1,3,4,5,6,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":724,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":741376,"data_stored":579193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8123,"internal_metadata":27582533},"hb_peers":[0,2,4,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,2,3,4,5,6,7,8,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[0,2,3,5,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8774,"internal_metadata":27581882},"hb_peers":[2,3,4,5,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[1,2,4,5,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[1,3,5,6,7,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8124,"internal_metadata":27582532},"hb_peers":[0,1,2,4,5,6,7,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,1,2,3,5,6,8,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,1,2,3,4,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":724,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":741376,"data_stored":579193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,1,2,3,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8780,"internal_metadata":27581876},"hb_peers":[0,1,2,3,4,6,7,8,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8780,"internal_metadata":27581876},"hb_peers":[0,1,3,4,5,6,7,8,9,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313221,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":276,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":282624,"data_stored":119932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8776,"internal_metadata":27581880},"hb_peers":[1,3,4,5,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313222,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[1,2,3,5,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T15:01:46.162 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.227+0000 7f76ac72c640 10 monclient: get_monmap_and_config 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.227+0000 7f76ac72c640 10 monclient: build_initial_monmap 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.227+0000 7f76ac72c640 10 monclient: monmap: 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:46.227726+0000 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:46.227726+0000 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.227+0000 7f76ac72c640 1 Processor -- start 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.227+0000 7f76ac72c640 1 -- start start 2026-02-25T15:01:46.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.227+0000 7f76ac72c640 10 monclient: init 2026-02-25T15:01:46.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.228+0000 7f76ac72c640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:46.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.228+0000 7f76ac72c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:46.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.228+0000 7f76ac72c640 10 monclient(hunting): picked mon.noname-c con 0x7f76a4058da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:46.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.228+0000 7f76ac72c640 10 monclient(hunting): picked mon.noname-a con 0x7f76a40b0410 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:46.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.228+0000 7f76ac72c640 10 monclient(hunting): picked mon.noname-b con 0x7f76a41626a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:46.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.228+0000 7f76ac72c640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a41655c0 con 0x7f76a40b0410 2026-02-25T15:01:46.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.228+0000 7f76ac72c640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a40b12b0 con 0x7f76a41626a0 2026-02-25T15:01:46.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.228+0000 7f76ac72c640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a40b1590 con 0x7f76a4058da0 2026-02-25T15:01:46.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.228+0000 7f76ac72c640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:46.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.228+0000 7f76ac72c640 10 monclient(hunting): authenticate will time out at 2995.828179s 2026-02-25T15:01:46.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.228+0000 7f76a9ca0640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f76a40b0410 0x7f76a4164eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57572/0 (socket says 10.20.193.90:57572) 2026-02-25T15:01:46.229 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.228+0000 7f76a9ca0640 1 -- 10.20.193.90:0/384439010 learned_addr learned my addr 10.20.193.90:0/384439010 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 1 -- 10.20.193.90:0/384439010 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3921661344 0 0) 0x7f76a41655c0 con 0x7f76a40b0410 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 10 monclient(hunting): _negotiate 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 10 monclient(hunting): my global_id is 4668 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 1 -- 10.20.193.90:0/384439010 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f768c003610 con 0x7f76a40b0410 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 1 -- 10.20.193.90:0/384439010 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2441027656 0 0) 0x7f76a40b1590 con 0x7f76a4058da0 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 10 monclient(hunting): _negotiate 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 10 monclient(hunting): my global_id is 4670 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 1 -- 10.20.193.90:0/384439010 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76a41655c0 con 0x7f76a4058da0 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 1 -- 10.20.193.90:0/384439010 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 659785893 0 0) 0x7f76a40b12b0 con 0x7f76a41626a0 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 10 monclient(hunting): _negotiate 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 10 monclient(hunting): my global_id is 4645 2026-02-25T15:01:46.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.229+0000 7f76a949f640 1 -- 10.20.193.90:0/384439010 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76a40b1590 con 0x7f76a41626a0 2026-02-25T15:01:46.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76a949f640 1 -- 10.20.193.90:0/384439010 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2404251332 0 0) 0x7f76a41655c0 con 0x7f76a4058da0 2026-02-25T15:01:46.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76a949f640 1 -- 10.20.193.90:0/384439010 >> v1:10.20.193.93:6789/0 conn(0x7f76a41626a0 legacy=0x7f76a41c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76a949f640 1 -- 10.20.193.90:0/384439010 >> v1:10.20.193.90:6789/0 conn(0x7f76a40b0410 legacy=0x7f76a4164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76a949f640 10 monclient: _finish_hunting 0 2026-02-25T15:01:46.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76a949f640 1 monclient: found mon.noname-c 2026-02-25T15:01:46.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76a949f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:46.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76a949f640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76a949f640 1 -- 10.20.193.90:0/384439010 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a40b1870 con 0x7f76a4058da0 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76a949f640 10 monclient: _finish_auth 0 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76a949f640 10 monclient: _check_auth_tickets 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76a949f640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76a949f640 1 -- 10.20.193.90:0/384439010 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7694002df0 con 0x7f76a4058da0 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76ac72c640 5 monclient: authenticate success, global_id 4670 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76ac72c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76a949f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76a949f640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76a949f640 10 monclient: dump: 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.231+0000 7f76ac72c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.232+0000 7f76a949f640 1 -- 10.20.193.90:0/384439010 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7694002fc0 con 0x7f76a4058da0 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.232+0000 7f76a949f640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.232+0000 7f76a949f640 1 -- 10.20.193.90:0/384439010 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7694003400 con 0x7f76a4058da0 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.232+0000 7f76a949f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.232+0000 7f76a949f640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.232+0000 7f76a949f640 10 monclient: dump: 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.232+0000 7f76ac72c640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.232+0000 7f76ac72c640 4 set_mon_vals no callback set 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.232+0000 7f76ac72c640 10 monclient: shutdown 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.232+0000 7f76ac72c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.232+0000 7f76ac72c640 1 -- 10.20.193.90:0/384439010 >> v1:10.20.193.94:6789/0 conn(0x7f76a4058da0 legacy=0x7f76a41647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.232+0000 7f76ac72c640 1 -- 10.20.193.90:0/384439010 shutdown_connections 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.232+0000 7f76ac72c640 1 -- 10.20.193.90:0/384439010 >> 10.20.193.90:0/384439010 conn(0x7f76a40b65f0 msgr2=0x7f76a40b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.232+0000 7f76ac72c640 1 -- 10.20.193.90:0/384439010 shutdown_connections 2026-02-25T15:01:46.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.232+0000 7f76ac72c640 1 -- 10.20.193.90:0/384439010 wait complete. 2026-02-25T15:01:46.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.233+0000 7f76ac72c640 10 monclient: build_initial_monmap 2026-02-25T15:01:46.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.233+0000 7f76ac72c640 10 monclient: monmap: 2026-02-25T15:01:46.235 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:46.235 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:46.235 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:46.235 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:46.235 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.233+0000 7f76ac72c640 1 librados: starting msgr at 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.233+0000 7f76ac72c640 1 librados: starting objecter 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.233+0000 7f76ac72c640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.233+0000 7f76ac72c640 1 Processor -- start 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.233+0000 7f76ac72c640 1 -- start start 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.233+0000 7f76ac72c640 1 librados: setting wanted keys 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.233+0000 7f76ac72c640 1 librados: calling monclient init 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.233+0000 7f76ac72c640 10 monclient: init 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f76ac72c640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f76ac72c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f76ac72c640 10 monclient(hunting): picked mon.noname-c con 0x7f76a4058da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f76ac72c640 10 monclient(hunting): picked mon.noname-a con 0x7f76a40b0410 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f76ac72c640 10 monclient(hunting): picked mon.noname-b con 0x7f76a41626a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f76ac72c640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a40a8050 con 0x7f76a40b0410 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f76ac72c640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a40a8330 con 0x7f76a41626a0 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f76ac72c640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a40a8610 con 0x7f76a4058da0 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f76ac72c640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f76ac72c640 10 monclient(hunting): authenticate will time out at 2995.834146s 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f76a9ca0640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f76a40b0410 0x7f76a4115c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57582/0 (socket says 10.20.193.90:57582) 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f76a9ca0640 1 -- 10.20.193.90:0/2609067442 learned_addr learned my addr 10.20.193.90:0/2609067442 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f768affd640 10 client.?.objecter ms_handle_connect 0x7f76a40b0410 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f768affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f768affd640 10 client.?.objecter ms_handle_connect 0x7f76a41626a0 2026-02-25T15:01:46.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.234+0000 7f768affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.235+0000 7f768affd640 10 client.?.objecter ms_handle_connect 0x7f76a4058da0 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.235+0000 7f768affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.235+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2985256962 0 0) 0x7f76a40a8050 con 0x7f76a40b0410 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.235+0000 7f768affd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.235+0000 7f768affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.235+0000 7f768affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.235+0000 7f768affd640 10 monclient(hunting): my global_id is 4671 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.235+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7674003610 con 0x7f76a40b0410 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.235+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3164813237 0 0) 0x7f76a40a8610 con 0x7f76a4058da0 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.235+0000 7f768affd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.235+0000 7f768affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.235+0000 7f768affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.235+0000 7f768affd640 10 monclient(hunting): my global_id is 4673 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.235+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76a40a8050 con 0x7f76a4058da0 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.236+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 397761491 0 0) 0x7f76a40a8330 con 0x7f76a41626a0 2026-02-25T15:01:46.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.236+0000 7f768affd640 10 monclient(hunting): _negotiate 2026-02-25T15:01:46.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.236+0000 7f768affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:46.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.236+0000 7f768affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:46.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.236+0000 7f768affd640 10 monclient(hunting): my global_id is 4648 2026-02-25T15:01:46.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.236+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76a40a8610 con 0x7f76a41626a0 2026-02-25T15:01:46.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.237+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 518547926 0 0) 0x7f7674003610 con 0x7f76a40b0410 2026-02-25T15:01:46.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.237+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f76a40a8330 con 0x7f76a40b0410 2026-02-25T15:01:46.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.237+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f76980030b0 con 0x7f76a40b0410 2026-02-25T15:01:46.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.237+0000 7f768affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:46.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.237+0000 7f768affd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.237+0000 7f768affd640 10 monclient(hunting): dump: 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.237+0000 7f768affd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.238+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 225078318 0 0) 0x7f76a40a8050 con 0x7f76a4058da0 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.238+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7674003610 con 0x7f76a4058da0 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.238+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7694003310 con 0x7f76a4058da0 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.238+0000 7f768affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.238+0000 7f768affd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.238+0000 7f768affd640 10 monclient(hunting): dump: 2026-02-25T15:01:46.239 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.238+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2024538374 0 0) 0x7f76a40a8610 con 0x7f76a41626a0 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.238+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f76a40a8050 con 0x7f76a41626a0 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.238+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f76a0003330 con 0x7f76a41626a0 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.238+0000 7f768affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.238+0000 7f768affd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.238+0000 7f768affd640 10 monclient(hunting): dump: 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:46.240 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3964776050 0 0) 0x7f76a40a8330 con 0x7f76a40b0410 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 >> v1:10.20.193.94:6789/0 conn(0x7f76a4058da0 legacy=0x7f76a40a7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 >> v1:10.20.193.93:6789/0 conn(0x7f76a41626a0 legacy=0x7f76a4135360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f768affd640 10 monclient: _finish_hunting 0 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f768affd640 1 monclient: found mon.a 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f768affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f768affd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a40af500 con 0x7f76a40b0410 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f768affd640 10 monclient: _finish_auth 0 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f768affd640 10 monclient: _check_auth_tickets 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f768affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f76ac72c640 5 monclient: authenticate success, global_id 4671 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f76ac72c640 10 monclient: _renew_subs 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f76ac72c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f76ac72c640 1 -- 10.20.193.90:0/2609067442 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f76a40af7e0 con 0x7f76a40b0410 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f76ac72c640 10 client.4671.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f76ac72c640 10 monclient: _renew_subs 2026-02-25T15:01:46.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f76ac72c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:46.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f76ac72c640 1 -- 10.20.193.90:0/2609067442 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f76a40af1b0 con 0x7f76a40b0410 2026-02-25T15:01:46.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f76ac72c640 1 librados: init done 2026-02-25T15:01:46.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f76ac72c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:46.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f76ac72c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:46.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f76ac72c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:01:46.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.239+0000 7f76ac72c640 1 -- 10.20.193.90:0/2609067442 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a40ad8f0 con 0x7f76a40b0410 2026-02-25T15:01:46.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.247+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f76980048d0 con 0x7f76a40b0410 2026-02-25T15:01:46.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.247+0000 7f768affd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:46.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.247+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7698004c20 con 0x7f76a40b0410 2026-02-25T15:01:46.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.247+0000 7f768affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:46.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.247+0000 7f768affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:46.248 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.247+0000 7f768affd640 10 monclient: dump: 2026-02-25T15:01:46.248 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:46.248 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:46.249 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.249 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.249 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:46.249 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:46.249 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:46.249 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:46.249 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:46.249 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.247+0000 7f76a8c9e640 4 set_mon_vals no callback set 2026-02-25T15:01:46.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.248+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f769801e100 con 0x7f76a40b0410 2026-02-25T15:01:46.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.248+0000 7f76aa4a1640 10 monclient: get_auth_request con 0x7f767407a040 auth_method 0 2026-02-25T15:01:46.249 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.249+0000 7f768affd640 10 client.4671.objecter ms_handle_connect 0x7f767407a040 2026-02-25T15:01:46.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.249+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(29..29 src has 1..29) ==== 8141+0+0 (unknown 2043172091 0 0) 0x7f7698095b10 con 0x7f76a40b0410 2026-02-25T15:01:46.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.249+0000 7f768affd640 10 client.4671.objecter ms_dispatch2 0x7f76a4000b90 osd_map(29..29 src has 1..29) 2026-02-25T15:01:46.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.249+0000 7f768affd640 3 client.4671.objecter handle_osd_map got epochs [29,29] > 0 2026-02-25T15:01:46.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.249+0000 7f768affd640 3 client.4671.objecter handle_osd_map decoding full epoch 29 2026-02-25T15:01:46.250 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.249+0000 7f768affd640 20 client.4671.objecter dump_active .. 0 homeless 2026-02-25T15:01:46.334 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.333+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f769805f470 con 0x7f76a40b0410 2026-02-25T15:01:46.334 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.333+0000 7f768affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:46.334 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.333+0000 7f768affd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:46.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.436+0000 7f76ac72c640 1 -- 10.20.193.90:0/2609067442 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76a40b6d40 con 0x7f767407a040 2026-02-25T15:01:46.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.436+0000 7f768affd640 1 -- 10.20.193.90:0/2609067442 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+42839 (unknown 2965378022 0 333748029) 0x7f76a40b6d40 con 0x7f767407a040 2026-02-25T15:01:46.438 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:01:46.438 INFO:teuthology.orchestra.run.trial090.stderr:dumped all 2026-02-25T15:01:46.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.437+0000 7f76ac72c640 10 librados: watch_flush enter 2026-02-25T15:01:46.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.437+0000 7f76ac72c640 10 librados: watch_flush exit 2026-02-25T15:01:46.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.437+0000 7f76ac72c640 20 client.4671.objecter shutdown clearing up homeless session... 2026-02-25T15:01:46.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.437+0000 7f76ac72c640 10 client.4671.objecter successfully canceled tick 2026-02-25T15:01:46.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.437+0000 7f76ac72c640 1 -- 10.20.193.90:0/2609067442 >> v1:10.20.193.189:6800/3647375982 conn(0x7f767407a040 legacy=0x7f767409a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.437+0000 7f76ac72c640 10 monclient: shutdown 2026-02-25T15:01:46.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.437+0000 7f76ac72c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:46.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.437+0000 7f76ac72c640 1 -- 10.20.193.90:0/2609067442 >> v1:10.20.193.90:6789/0 conn(0x7f76a40b0410 legacy=0x7f76a4115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.437+0000 7f76ac72c640 1 -- 10.20.193.90:0/2609067442 shutdown_connections 2026-02-25T15:01:46.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.437+0000 7f76ac72c640 1 -- 10.20.193.90:0/2609067442 >> 10.20.193.90:0/2609067442 conn(0x7f76a40b65f0 msgr2=0x7f76a4165450 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:46.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.437+0000 7f76ac72c640 1 -- 10.20.193.90:0/2609067442 shutdown_connections 2026-02-25T15:01:46.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.437+0000 7f76ac72c640 1 -- 10.20.193.90:0/2609067442 wait complete. 2026-02-25T15:01:46.439 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.437+0000 7f76ac72c640 1 librados: shutdown 2026-02-25T15:01:46.446 INFO:teuthology.orchestra.run.trial090.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2026-02-25T15:01:45.418776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":436624,"kb_used_data":5264,"kb_used_omap":132,"kb_used_meta":430971,"kb_avail":1509512816,"statfs":{"total":1546188226560,"available":1545741123584,"internally_reserved":0,"allocated":5390336,"data_stored":2837206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":135852,"internal_metadata":441314644},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131107,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":18,"num_write_kb":264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.353284"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:40.614364+0000","last_change":"2026-02-25T15:01:40.617020+0000","last_active":"2026-02-25T15:01:40.614364+0000","last_peered":"2026-02-25T15:01:40.614364+0000","last_clean":"2026-02-25T15:01:40.614364+0000","last_became_active":"2026-02-25T15:01:38.997857+0000","last_became_peered":"2026-02-25T15:01:38.997857+0000","last_unstale":"2026-02-25T15:01:40.614364+0000","last_undegraded":"2026-02-25T15:01:40.614364+0000","last_fullsized":"2026-02-25T15:01:40.614364+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:49:56.506302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062975700999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14],"acting":[11,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:41.108756+0000","last_change":"2026-02-25T15:01:41.128386+0000","last_active":"2026-02-25T15:01:41.108756+0000","last_peered":"2026-02-25T15:01:41.108756+0000","last_clean":"2026-02-25T15:01:41.108756+0000","last_became_active":"2026-02-25T15:01:38.782341+0000","last_became_peered":"2026-02-25T15:01:38.782341+0000","last_unstale":"2026-02-25T15:01:41.108756+0000","last_undegraded":"2026-02-25T15:01:41.108756+0000","last_fullsized":"2026-02-25T15:01:41.108756+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:07:32.765627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031008292,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,3],"acting":[12,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:41.135348+0000","last_change":"2026-02-25T15:01:41.143783+0000","last_active":"2026-02-25T15:01:41.135348+0000","last_peered":"2026-02-25T15:01:41.135348+0000","last_clean":"2026-02-25T15:01:41.135348+0000","last_became_active":"2026-02-25T15:01:38.731569+0000","last_became_peered":"2026-02-25T15:01:38.731569+0000","last_unstale":"2026-02-25T15:01:41.135348+0000","last_undegraded":"2026-02-25T15:01:41.135348+0000","last_fullsized":"2026-02-25T15:01:41.135348+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:00:25.099174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037093318,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15],"acting":[7,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:41.116719+0000","last_change":"2026-02-25T15:01:41.139794+0000","last_active":"2026-02-25T15:01:41.116719+0000","last_peered":"2026-02-25T15:01:41.116719+0000","last_clean":"2026-02-25T15:01:41.116719+0000","last_became_active":"2026-02-25T15:01:38.780556+0000","last_became_peered":"2026-02-25T15:01:38.780556+0000","last_unstale":"2026-02-25T15:01:41.116719+0000","last_undegraded":"2026-02-25T15:01:41.116719+0000","last_fullsized":"2026-02-25T15:01:41.116719+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:57:49.842137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041710525999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:40.608781+0000","last_change":"2026-02-25T15:01:40.608781+0000","last_active":"2026-02-25T15:01:40.608781+0000","last_peered":"2026-02-25T15:01:40.608781+0000","last_clean":"2026-02-25T15:01:40.608781+0000","last_became_active":"2026-02-25T15:01:38.743941+0000","last_became_peered":"2026-02-25T15:01:38.743941+0000","last_unstale":"2026-02-25T15:01:40.608781+0000","last_undegraded":"2026-02-25T15:01:40.608781+0000","last_fullsized":"2026-02-25T15:01:40.608781+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:53:32.920667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.061484652000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13],"acting":[11,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:41.108353+0000","last_change":"2026-02-25T15:01:41.129594+0000","last_active":"2026-02-25T15:01:41.108353+0000","last_peered":"2026-02-25T15:01:41.108353+0000","last_clean":"2026-02-25T15:01:41.108353+0000","last_became_active":"2026-02-25T15:01:38.861789+0000","last_became_peered":"2026-02-25T15:01:38.861789+0000","last_unstale":"2026-02-25T15:01:41.108353+0000","last_undegraded":"2026-02-25T15:01:41.108353+0000","last_fullsized":"2026-02-25T15:01:41.108353+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:36:46.424677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032448785000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:41.097316+0000","last_change":"2026-02-25T15:01:41.116971+0000","last_active":"2026-02-25T15:01:41.097316+0000","last_peered":"2026-02-25T15:01:41.097316+0000","last_clean":"2026-02-25T15:01:41.097316+0000","last_became_active":"2026-02-25T15:01:38.737942+0000","last_became_peered":"2026-02-25T15:01:38.737942+0000","last_unstale":"2026-02-25T15:01:41.097316+0000","last_undegraded":"2026-02-25T15:01:41.097316+0000","last_fullsized":"2026-02-25T15:01:41.097316+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:51:33.138764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031295004000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13],"acting":[0,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:40.611559+0000","last_change":"2026-02-25T15:01:40.614625+0000","last_active":"2026-02-25T15:01:40.611559+0000","last_peered":"2026-02-25T15:01:40.611559+0000","last_clean":"2026-02-25T15:01:40.611559+0000","last_became_active":"2026-02-25T15:01:38.791923+0000","last_became_peered":"2026-02-25T15:01:38.791923+0000","last_unstale":"2026-02-25T15:01:40.611559+0000","last_undegraded":"2026-02-25T15:01:40.611559+0000","last_fullsized":"2026-02-25T15:01:40.611559+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:51:10.001377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062725172999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5],"acting":[11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"24'32","reported_seq":65,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-25T15:01:41.251864+0000","last_change":"2026-02-25T15:01:30.868615+0000","last_active":"2026-02-25T15:01:41.251864+0000","last_peered":"2026-02-25T15:01:41.251864+0000","last_clean":"2026-02-25T15:01:41.251864+0000","last_became_active":"2026-02-25T15:01:30.851092+0000","last_became_peered":"2026-02-25T15:01:30.851092+0000","last_unstale":"2026-02-25T15:01:41.251864+0000","last_undegraded":"2026-02-25T15:01:41.251864+0000","last_fullsized":"2026-02-25T15:01:41.251864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:29.405841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:29.405841+0000","last_clean_scrub_stamp":"2026-02-25T15:01:29.405841+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T02:31:46.315230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3],"acting":[8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":276,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":282624,"data_stored":119932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[0,2,3,4,6,7,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8120,"internal_metadata":27582536},"hb_peers":[0,1,3,4,5,6,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":724,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":741376,"data_stored":579193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8123,"internal_metadata":27582533},"hb_peers":[0,2,4,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,2,3,4,5,6,7,8,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[0,2,3,5,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8774,"internal_metadata":27581882},"hb_peers":[2,3,4,5,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[1,2,4,5,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[1,3,5,6,7,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8124,"internal_metadata":27582532},"hb_peers":[0,1,2,4,5,6,7,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,1,2,3,5,6,8,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,1,2,3,4,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":724,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":741376,"data_stored":579193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,1,2,3,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8780,"internal_metadata":27581876},"hb_peers":[0,1,2,3,4,6,7,8,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8780,"internal_metadata":27581876},"hb_peers":[0,1,3,4,5,6,7,8,9,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313221,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":276,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":282624,"data_stored":119932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8776,"internal_metadata":27581880},"hb_peers":[1,3,4,5,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313222,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":272,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":278528,"data_stored":119913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[1,2,3,5,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T15:01:46.447 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-25T15:01:46.447 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-25T15:01:46.447 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-25T15:01:46.447 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-25T15:01:46.514 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.513+0000 7f3482d45640 10 monclient: get_monmap_and_config 2026-02-25T15:01:46.514 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.513+0000 7f3482d45640 10 monclient: build_initial_monmap 2026-02-25T15:01:46.514 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.513+0000 7f3482d45640 10 monclient: monmap: 2026-02-25T15:01:46.514 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:46.514 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:46.514 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:01:46.514220+0000 2026-02-25T15:01:46.514 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:01:46.514220+0000 2026-02-25T15:01:46.514 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:46.514 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:46.515 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:46.515 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:46.515 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:46.515 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.513+0000 7f3482d45640 1 Processor -- start 2026-02-25T15:01:46.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.514+0000 7f3482d45640 1 -- start start 2026-02-25T15:01:46.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.514+0000 7f3482d45640 10 monclient: init 2026-02-25T15:01:46.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.514+0000 7f3482d45640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:46.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.514+0000 7f3482d45640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:01:46.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.514+0000 7f3482d45640 10 monclient(hunting): picked mon.noname-b con 0x7f347c058da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:46.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.514+0000 7f3482d45640 10 monclient(hunting): picked mon.noname-a con 0x7f347c0b0410 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:46.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.514+0000 7f3482d45640 10 monclient(hunting): picked mon.noname-c con 0x7f347c1626a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:46.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.514+0000 7f3482d45640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f347c1655c0 con 0x7f347c0b0410 2026-02-25T15:01:46.515 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.514+0000 7f3482d45640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f347c0b12b0 con 0x7f347c058da0 2026-02-25T15:01:46.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.514+0000 7f3482d45640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f347c0b1590 con 0x7f347c1626a0 2026-02-25T15:01:46.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.514+0000 7f3482d45640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:46.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.514+0000 7f3482d45640 10 monclient(hunting): authenticate will time out at 2996.114556s 2026-02-25T15:01:46.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.515+0000 7f3480aba640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f347c058da0 0x7f347c1647a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:51338/0 (socket says 10.20.193.90:51338) 2026-02-25T15:01:46.516 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.515+0000 7f3480aba640 1 -- 10.20.193.90:0/4060906518 learned_addr learned my addr 10.20.193.90:0/4060906518 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:46.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.516+0000 7f34737fe640 1 -- 10.20.193.90:0/4060906518 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1903794727 0 0) 0x7f347c0b12b0 con 0x7f347c058da0 2026-02-25T15:01:46.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.516+0000 7f34737fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:46.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.516+0000 7f34737fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:46.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.516+0000 7f34737fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:46.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.516+0000 7f34737fe640 10 monclient(hunting): my global_id is 4651 2026-02-25T15:01:46.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.516+0000 7f34737fe640 1 -- 10.20.193.90:0/4060906518 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3468003610 con 0x7f347c058da0 2026-02-25T15:01:46.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.516+0000 7f34737fe640 1 -- 10.20.193.90:0/4060906518 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4085119430 0 0) 0x7f347c0b1590 con 0x7f347c1626a0 2026-02-25T15:01:46.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.516+0000 7f34737fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:46.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.516+0000 7f34737fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:46.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.516+0000 7f34737fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:46.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.516+0000 7f34737fe640 10 monclient(hunting): my global_id is 4676 2026-02-25T15:01:46.517 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.516+0000 7f34737fe640 1 -- 10.20.193.90:0/4060906518 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f347c0b12b0 con 0x7f347c1626a0 2026-02-25T15:01:46.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 1 -- 10.20.193.90:0/4060906518 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1311701925 0 0) 0x7f347c1655c0 con 0x7f347c0b0410 2026-02-25T15:01:46.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 10 monclient(hunting): _negotiate 2026-02-25T15:01:46.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:46.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:46.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 10 monclient(hunting): my global_id is 4674 2026-02-25T15:01:46.518 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 1 -- 10.20.193.90:0/4060906518 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f347c0b1590 con 0x7f347c0b0410 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 1 -- 10.20.193.90:0/4060906518 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2426246090 0 0) 0x7f3468003610 con 0x7f347c058da0 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 1 -- 10.20.193.90:0/4060906518 >> v1:10.20.193.94:6789/0 conn(0x7f347c1626a0 legacy=0x7f347c1c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 1 -- 10.20.193.90:0/4060906518 >> v1:10.20.193.90:6789/0 conn(0x7f347c0b0410 legacy=0x7f347c164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 10 monclient: _finish_hunting 0 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 1 monclient: found mon.noname-b 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 1 -- 10.20.193.90:0/4060906518 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c0b1870 con 0x7f347c058da0 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 10 monclient: _finish_auth 0 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 10 monclient: _check_auth_tickets 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f3482d45640 5 monclient: authenticate success, global_id 4651 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f3482d45640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 1 -- 10.20.193.90:0/4060906518 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f346c002df0 con 0x7f347c058da0 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.517+0000 7f34737fe640 10 monclient: dump: 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:46.519 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:46.520 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:46.520 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.518+0000 7f3482d45640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:01:46.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.518+0000 7f34737fe640 1 -- 10.20.193.90:0/4060906518 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f346c003040 con 0x7f347c058da0 2026-02-25T15:01:46.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.518+0000 7f34737fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:46.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.518+0000 7f34737fe640 1 -- 10.20.193.90:0/4060906518 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f346c003440 con 0x7f347c058da0 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.518+0000 7f3482d45640 10 monclient: get_monmap_and_config success 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.518+0000 7f3482d45640 4 set_mon_vals no callback set 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.518+0000 7f34737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.518+0000 7f34737fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.518+0000 7f34737fe640 10 monclient: dump: 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 10 monclient: shutdown 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 1 -- 10.20.193.90:0/4060906518 >> v1:10.20.193.93:6789/0 conn(0x7f347c058da0 legacy=0x7f347c1647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 1 -- 10.20.193.90:0/4060906518 shutdown_connections 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 1 -- 10.20.193.90:0/4060906518 >> 10.20.193.90:0/4060906518 conn(0x7f347c0b65f0 msgr2=0x7f347c0b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 1 -- 10.20.193.90:0/4060906518 shutdown_connections 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 1 -- 10.20.193.90:0/4060906518 wait complete. 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 10 monclient: build_initial_monmap 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 10 monclient: monmap: 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:01:46.521 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:01:46.522 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:01:46.522 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:01:46.522 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 1 librados: starting msgr at 2026-02-25T15:01:46.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 1 librados: starting objecter 2026-02-25T15:01:46.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:01:46.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 1 Processor -- start 2026-02-25T15:01:46.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 1 -- start start 2026-02-25T15:01:46.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 1 librados: setting wanted keys 2026-02-25T15:01:46.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 1 librados: calling monclient init 2026-02-25T15:01:46.522 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.519+0000 7f3482d45640 10 monclient: init 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f3482d45640 10 monclient: _reopen_session rank -1 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f3482d45640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f3482d45640 10 monclient(hunting): picked mon.noname-c con 0x7f347c058da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f3482d45640 10 monclient(hunting): picked mon.noname-a con 0x7f347c0b0410 addr v1:10.20.193.90:6789/0 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f3482d45640 10 monclient(hunting): picked mon.noname-b con 0x7f347c1626a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f3482d45640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f347c0a81c0 con 0x7f347c0b0410 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f3482d45640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f347c05bce0 con 0x7f347c1626a0 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f3482d45640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f347c05bfc0 con 0x7f347c058da0 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f3482d45640 10 monclient(hunting): _renew_subs 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f3482d45640 10 monclient(hunting): authenticate will time out at 2996.120277s 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f3473fff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f347c0b0410 0x7f347c0a87b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:57600/0 (socket says 10.20.193.90:57600) 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f3473fff640 1 -- 10.20.193.90:0/1972242472 learned_addr learned my addr 10.20.193.90:0/1972242472 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f34717fa640 10 client.?.objecter ms_handle_connect 0x7f347c0b0410 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f34717fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f34717fa640 10 client.?.objecter ms_handle_connect 0x7f347c058da0 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.520+0000 7f34717fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:46.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 10 client.?.objecter ms_handle_connect 0x7f347c1626a0 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3876614732 0 0) 0x7f347c0a81c0 con 0x7f347c0b0410 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 10 monclient(hunting): my global_id is 4677 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f344c003610 con 0x7f347c0b0410 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4255170892 0 0) 0x7f347c05bfc0 con 0x7f347c058da0 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 10 monclient(hunting): my global_id is 4679 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f347c0a81c0 con 0x7f347c058da0 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3288108970 0 0) 0x7f347c05bce0 con 0x7f347c1626a0 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 10 monclient(hunting): _negotiate 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 10 monclient(hunting): my global_id is 4654 2026-02-25T15:01:46.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.521+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f347c05bfc0 con 0x7f347c1626a0 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.523+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 847178727 0 0) 0x7f347c0a81c0 con 0x7f347c058da0 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.523+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f347c05bce0 con 0x7f347c058da0 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.523+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f346c003390 con 0x7f347c058da0 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.523+0000 7f34717fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.523+0000 7f34717fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.523+0000 7f34717fa640 10 monclient(hunting): dump: 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.523+0000 7f34717fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.524+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 651534339 0 0) 0x7f344c003610 con 0x7f347c0b0410 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.524+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f347c0a81c0 con 0x7f347c0b0410 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.524+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3464003170 con 0x7f347c0b0410 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.524+0000 7f34717fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.524+0000 7f34717fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.524+0000 7f34717fa640 10 monclient(hunting): dump: 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:46.525 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.526 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.526 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:46.526 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:46.526 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:46.526 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:46.526 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:46.526 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.524+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1193931291 0 0) 0x7f347c05bfc0 con 0x7f347c1626a0 2026-02-25T15:01:46.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.524+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f344c003610 con 0x7f347c1626a0 2026-02-25T15:01:46.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.524+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3478003290 con 0x7f347c1626a0 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.524+0000 7f34717fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.524+0000 7f34717fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.524+0000 7f34717fa640 10 monclient(hunting): dump: 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2230944800 0 0) 0x7f347c05bce0 con 0x7f347c058da0 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 >> v1:10.20.193.93:6789/0 conn(0x7f347c1626a0 legacy=0x7f347c1faca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 >> v1:10.20.193.90:6789/0 conn(0x7f347c0b0410 legacy=0x7f347c0a87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f34717fa640 10 monclient: _finish_hunting 0 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f34717fa640 1 monclient: found mon.c 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f34717fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f34717fa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c05c2a0 con 0x7f347c058da0 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f34717fa640 10 monclient: _finish_auth 0 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f34717fa640 10 monclient: _check_auth_tickets 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f34717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f3482d45640 5 monclient: authenticate success, global_id 4679 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f3482d45640 10 monclient: _renew_subs 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f3482d45640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f3482d45640 1 -- 10.20.193.90:0/1972242472 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f347c05c580 con 0x7f347c058da0 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f3482d45640 10 client.4679.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:01:46.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f3482d45640 10 monclient: _renew_subs 2026-02-25T15:01:46.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f3482d45640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:46.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f3482d45640 1 -- 10.20.193.90:0/1972242472 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f347c05cb40 con 0x7f347c058da0 2026-02-25T15:01:46.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.525+0000 7f3482d45640 1 librados: init done 2026-02-25T15:01:46.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.526+0000 7f3482d45640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:46.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.526+0000 7f3482d45640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:46.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.526+0000 7f3482d45640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:46.528 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.526+0000 7f3482d45640 1 -- 10.20.193.90:0/1972242472 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347c0b28d0 con 0x7f347c058da0 2026-02-25T15:01:46.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.617+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f346c002ff0 con 0x7f347c058da0 2026-02-25T15:01:46.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.617+0000 7f34717fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:01:46.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.617+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f346c004ce0 con 0x7f347c058da0 2026-02-25T15:01:46.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.617+0000 7f34717fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:01:46.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.617+0000 7f34717fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:01:46.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.617+0000 7f3472ffd640 4 set_mon_vals no callback set 2026-02-25T15:01:46.617 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.617+0000 7f34717fa640 10 monclient: dump: 2026-02-25T15:01:46.618 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:01:46.618 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:01:46.618 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.618 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:01:46.618 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:01:46.618 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:01:46.618 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:01:46.618 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:01:46.618 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:01:46.618 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:01:46.618 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.617+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f346c01e1c0 con 0x7f347c058da0 2026-02-25T15:01:46.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.618+0000 7f3473fff640 10 monclient: get_auth_request con 0x7f344c07a0c0 auth_method 0 2026-02-25T15:01:46.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.618+0000 7f34717fa640 10 client.4679.objecter ms_handle_connect 0x7f344c07a0c0 2026-02-25T15:01:46.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.618+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(29..29 src has 1..29) ==== 8141+0+0 (unknown 2043172091 0 0) 0x7f346c095a60 con 0x7f347c058da0 2026-02-25T15:01:46.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.618+0000 7f34717fa640 10 client.4679.objecter ms_dispatch2 0x7f347c000b90 osd_map(29..29 src has 1..29) 2026-02-25T15:01:46.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.618+0000 7f34717fa640 3 client.4679.objecter handle_osd_map got epochs [29,29] > 0 2026-02-25T15:01:46.619 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.618+0000 7f34717fa640 3 client.4679.objecter handle_osd_map decoding full epoch 29 2026-02-25T15:01:46.620 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.618+0000 7f34717fa640 20 client.4679.objecter dump_active .. 0 homeless 2026-02-25T15:01:46.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.620+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f346c05f3c0 con 0x7f347c058da0 2026-02-25T15:01:46.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.620+0000 7f34717fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:01:46.621 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.620+0000 7f34717fa640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:01:46.756 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.754+0000 7f3482d45640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-25T15:01:46.756 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.754+0000 7f3482d45640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-25T15:01:46.756 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.754+0000 7f3482d45640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:01:46.756 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.754+0000 7f3482d45640 1 -- 10.20.193.90:0/1972242472 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f347c0b25f0 con 0x7f347c058da0 2026-02-25T15:01:46.759 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.758+0000 7f34717fa640 1 -- 10.20.193.90:0/1972242472 <== mon.2 v1:10.20.193.94:6789/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (unknown 3730786198 0 4185958460) 0x7f346c05ecc0 con 0x7f347c058da0 2026-02-25T15:01:46.759 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.758+0000 7f34717fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-25T15:01:46.760 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.758+0000 7f34717fa640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:01:46.760 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:01:46.760 INFO:teuthology.orchestra.run.trial090.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-25T15:01:46.761 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.759+0000 7f3482d45640 10 librados: watch_flush enter 2026-02-25T15:01:46.761 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.759+0000 7f3482d45640 10 librados: watch_flush exit 2026-02-25T15:01:46.761 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.759+0000 7f3482d45640 20 client.4679.objecter shutdown clearing up homeless session... 2026-02-25T15:01:46.762 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.759+0000 7f3482d45640 10 client.4679.objecter successfully canceled tick 2026-02-25T15:01:46.762 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.759+0000 7f3482d45640 1 -- 10.20.193.90:0/1972242472 >> v1:10.20.193.189:6800/3647375982 conn(0x7f344c07a0c0 legacy=0x7f344c09a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.762 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.759+0000 7f3482d45640 10 monclient: shutdown 2026-02-25T15:01:46.762 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.759+0000 7f3482d45640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:01:46.762 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.759+0000 7f3482d45640 1 -- 10.20.193.90:0/1972242472 >> v1:10.20.193.94:6789/0 conn(0x7f347c058da0 legacy=0x7f347c0a7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:01:46.762 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.759+0000 7f3482d45640 1 -- 10.20.193.90:0/1972242472 shutdown_connections 2026-02-25T15:01:46.762 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.759+0000 7f3482d45640 1 -- 10.20.193.90:0/1972242472 >> 10.20.193.90:0/1972242472 conn(0x7f347c0b65f0 msgr2=0x7f347c055980 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:01:46.762 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.759+0000 7f3482d45640 1 -- 10.20.193.90:0/1972242472 shutdown_connections 2026-02-25T15:01:46.762 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.759+0000 7f3482d45640 1 -- 10.20.193.90:0/1972242472 wait complete. 2026-02-25T15:01:46.762 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:01:46.759+0000 7f3482d45640 1 librados: shutdown 2026-02-25T15:01:46.770 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-25T15:01:46.770 INFO:teuthology.run_tasks:Running task workunit... 2026-02-25T15:01:46.786 INFO:tasks.workunit:Pulling workunits from ref c824a7fa29b067db9ce5a381f59b175918299919 2026-02-25T15:01:46.786 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-25T15:01:46.787 DEBUG:teuthology.orchestra.run.trial189:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-25T15:01:46.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T15:01:46.806 INFO:teuthology.orchestra.run.trial189.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-25T15:01:46.806 DEBUG:teuthology.orchestra.run.trial189:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-25T15:01:46.862 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-25T15:01:46.862 DEBUG:teuthology.orchestra.run.trial189:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-25T15:01:46.917 INFO:tasks.workunit:timeout=6h 2026-02-25T15:01:46.917 INFO:tasks.workunit:cleanup=True 2026-02-25T15:01:46.917 DEBUG:teuthology.orchestra.run.trial189:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout c824a7fa29b067db9ce5a381f59b175918299919 2026-02-25T15:01:46.979 INFO:tasks.workunit.client.0.trial189.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-25T15:01:50.359 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:01:03.867 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:01:50.382 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:01:03.851 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:01:50.388 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:01:03.895 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:01:50.408 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:01:03.911 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:01:50.409 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:01:03.878 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:01:50.434 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:01:03.937 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:01:50.509 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:01:04.016 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:01:50.530 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:01:03.999 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:01:50.553 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:01:04.056 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:10.168 INFO:tasks.workunit.client.0.trial189.stderr:Note: switching to 'c824a7fa29b067db9ce5a381f59b175918299919'. 2026-02-25T15:02:10.168 INFO:tasks.workunit.client.0.trial189.stderr: 2026-02-25T15:02:10.168 INFO:tasks.workunit.client.0.trial189.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-25T15:02:10.168 INFO:tasks.workunit.client.0.trial189.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-25T15:02:10.168 INFO:tasks.workunit.client.0.trial189.stderr:state without impacting any branches by switching back to a branch. 2026-02-25T15:02:10.168 INFO:tasks.workunit.client.0.trial189.stderr: 2026-02-25T15:02:10.168 INFO:tasks.workunit.client.0.trial189.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-25T15:02:10.168 INFO:tasks.workunit.client.0.trial189.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-25T15:02:10.169 INFO:tasks.workunit.client.0.trial189.stderr: 2026-02-25T15:02:10.169 INFO:tasks.workunit.client.0.trial189.stderr: git switch -c 2026-02-25T15:02:10.169 INFO:tasks.workunit.client.0.trial189.stderr: 2026-02-25T15:02:10.169 INFO:tasks.workunit.client.0.trial189.stderr:Or undo this operation with: 2026-02-25T15:02:10.169 INFO:tasks.workunit.client.0.trial189.stderr: 2026-02-25T15:02:10.169 INFO:tasks.workunit.client.0.trial189.stderr: git switch - 2026-02-25T15:02:10.169 INFO:tasks.workunit.client.0.trial189.stderr: 2026-02-25T15:02:10.169 INFO:tasks.workunit.client.0.trial189.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-25T15:02:10.169 INFO:tasks.workunit.client.0.trial189.stderr: 2026-02-25T15:02:10.169 INFO:tasks.workunit.client.0.trial189.stderr:HEAD is now at c824a7fa29b Merge PR #65949 into tentacle 2026-02-25T15:02:10.173 DEBUG:teuthology.orchestra.run.trial189:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-25T15:02:10.231 INFO:tasks.workunit.client.0.trial189.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-25T15:02:10.234 INFO:tasks.workunit.client.0.trial189.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-25T15:02:10.234 INFO:tasks.workunit.client.0.trial189.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-25T15:02:10.322 INFO:tasks.workunit.client.0.trial189.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-25T15:02:10.354 INFO:tasks.workunit.client.0.trial189.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-25T15:02:10.360 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:01:23.867 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:10.361 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:01:23.869 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:10.380 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:01:23.887 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:10.380 INFO:tasks.workunit.client.0.trial189.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-25T15:02:10.382 INFO:tasks.workunit.client.0.trial189.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-25T15:02:10.382 INFO:tasks.workunit.client.0.trial189.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-25T15:02:10.387 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:01:23.890 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:10.388 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:01:23.891 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:10.389 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:01:23.859 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:10.390 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:01:23.860 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:10.406 INFO:tasks.workunit.client.0.trial189.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-25T15:02:10.406 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:01:23.909 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:10.408 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:01:23.877 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:10.410 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-25T15:02:10.410 DEBUG:teuthology.orchestra.run.trial189:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-25T15:02:10.468 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2026-02-25T15:02:10.470 INFO:tasks.workunit:Running workunit rados/test.sh... 2026-02-25T15:02:10.471 DEBUG:teuthology.orchestra.run.trial189:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c824a7fa29b067db9ce5a381f59b175918299919 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2026-02-25T15:02:10.529 INFO:tasks.workunit.client.0.trial189.stderr:+ parallel=1 2026-02-25T15:02:10.530 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' '' = --serial ']' 2026-02-25T15:02:10.530 INFO:tasks.workunit.client.0.trial189.stderr:+ crimson=0 2026-02-25T15:02:10.530 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' '' = --crimson ']' 2026-02-25T15:02:10.530 INFO:tasks.workunit.client.0.trial189.stderr:+ color= 2026-02-25T15:02:10.530 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -t 1 ']' 2026-02-25T15:02:10.530 INFO:tasks.workunit.client.0.trial189.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2026-02-25T15:02:10.530 INFO:tasks.workunit.client.0.trial189.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2026-02-25T15:02:10.530 INFO:tasks.workunit.client.0.trial189.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2026-02-25T15:02:10.531 INFO:tasks.workunit.client.0.trial189.stderr:+ declare -A pids 2026-02-25T15:02:10.531 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.532 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.532 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_aio 2026-02-25T15:02:10.532 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_aio' 2026-02-25T15:02:10.532 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_aio 2026-02-25T15:02:10.532 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.533 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_aio 2026-02-25T15:02:10.533 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49150 2026-02-25T15:02:10.533 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_aio on pid 49150' 2026-02-25T15:02:10.533 INFO:tasks.workunit.client.0.trial189.stdout:test api_aio on pid 49150 2026-02-25T15:02:10.535 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49150 2026-02-25T15:02:10.536 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.536 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.536 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2026-02-25T15:02:10.536 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_aio_pp 2026-02-25T15:02:10.536 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_aio_pp' 2026-02-25T15:02:10.536 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_aio_pp 2026-02-25T15:02:10.536 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.536 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.536 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.536 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.536 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.536 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.536 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.537 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.537 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.537 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.537 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.537 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.537 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.537 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.537 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.537 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.537 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.537 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.537 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.537 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.537 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_aio_pp 2026-02-25T15:02:10.537 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49157 2026-02-25T15:02:10.538 INFO:tasks.workunit.client.0.trial189.stdout:test api_aio_pp on pid 49157 2026-02-25T15:02:10.540 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_aio_pp on pid 49157' 2026-02-25T15:02:10.540 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49157 2026-02-25T15:02:10.540 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_io 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_io' 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_io 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.541 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.542 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.542 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.542 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.542 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.542 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.542 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.542 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.542 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.542 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.542 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.542 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.542 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.542 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.542 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.543 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.543 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.543 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.543 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.543 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.543 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.543 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.543 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.543 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.543 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.543 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.543 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.543 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.543 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.543 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_io 2026-02-25T15:02:10.544 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49167 2026-02-25T15:02:10.544 INFO:tasks.workunit.client.0.trial189.stdout:test api_io on pid 49167 2026-02-25T15:02:10.547 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_io on pid 49167' 2026-02-25T15:02:10.547 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49167 2026-02-25T15:02:10.547 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.547 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.547 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2026-02-25T15:02:10.547 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_io_pp 2026-02-25T15:02:10.547 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_io_pp' 2026-02-25T15:02:10.547 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.547 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.547 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_io_pp 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.548 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.549 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_io_pp 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49178 2026-02-25T15:02:10.550 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_io_pp on pid 49178' 2026-02-25T15:02:10.551 INFO:tasks.workunit.client.0.trial189.stdout:test api_io_pp on pid 49178 2026-02-25T15:02:10.555 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49178 2026-02-25T15:02:10.555 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.555 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.555 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2026-02-25T15:02:10.555 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_asio 2026-02-25T15:02:10.555 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_asio' 2026-02-25T15:02:10.555 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_asio 2026-02-25T15:02:10.555 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.555 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.555 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.555 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.556 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.557 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.558 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_asio 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.559 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.560 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.560 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.560 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.560 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.560 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.560 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49193 2026-02-25T15:02:10.560 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_asio on pid 49193' 2026-02-25T15:02:10.560 INFO:tasks.workunit.client.0.trial189.stdout:test api_asio on pid 49193 2026-02-25T15:02:10.564 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49193 2026-02-25T15:02:10.564 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.564 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.564 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.564 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_list 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_list' 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_list 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.565 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.566 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.567 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49150/exe ']' 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.568 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.569 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.569 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.569 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49150/exe 2026-02-25T15:02:10.569 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_list 2026-02-25T15:02:10.569 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49209 2026-02-25T15:02:10.569 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_list on pid 49209' 2026-02-25T15:02:10.569 INFO:tasks.workunit.client.0.trial189.stdout:test api_list on pid 49209 2026-02-25T15:02:10.573 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49209 2026-02-25T15:02:10.573 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.573 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.573 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2026-02-25T15:02:10.573 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.573 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.573 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.573 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_lock 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.574 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49157/exe ']' 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_lock' 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_lock 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49157/exe 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.575 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.576 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_aio.log 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_aio: /' 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.577 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_lock 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.578 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49230 2026-02-25T15:02:10.579 INFO:tasks.workunit.client.0.trial189.stdout:test api_lock on pid 49230 2026-02-25T15:02:10.582 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_lock on pid 49230' 2026-02-25T15:02:10.582 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49230 2026-02-25T15:02:10.582 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.582 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.582 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2026-02-25T15:02:10.582 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2026-02-25T15:02:10.582 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.582 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.582 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.582 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.582 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.582 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_lock_pp 2026-02-25T15:02:10.582 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_aio_pp.log 2026-02-25T15:02:10.582 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_aio_pp: /' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_lock_pp' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_lock_pp 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.583 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.584 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.585 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49167/exe ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49167/exe 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_lock_pp 2026-02-25T15:02:10.586 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49251 2026-02-25T15:02:10.587 INFO:tasks.workunit.client.0.trial189.stdout:test api_lock_pp on pid 49251 2026-02-25T15:02:10.590 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_lock_pp on pid 49251' 2026-02-25T15:02:10.590 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49251 2026-02-25T15:02:10.590 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.590 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.590 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2026-02-25T15:02:10.590 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.590 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_misc 2026-02-25T15:02:10.590 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_misc' 2026-02-25T15:02:10.590 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.590 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.590 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.590 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.590 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.590 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_misc 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49178/exe ']' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.591 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49178/exe 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.592 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_io.log 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_io: /' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.593 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.594 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.594 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.594 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.594 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.594 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.594 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_misc 2026-02-25T15:02:10.594 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49270 2026-02-25T15:02:10.594 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_misc on pid 49270' 2026-02-25T15:02:10.594 INFO:tasks.workunit.client.0.trial189.stdout:test api_misc on pid 49270 2026-02-25T15:02:10.598 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49270 2026-02-25T15:02:10.598 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.598 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_misc_pp 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_misc_pp' 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_io_pp.log 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_io_pp: /' 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.599 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_misc_pp 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.600 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.601 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49193/exe ']' 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.602 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.603 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.604 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.604 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.604 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49193/exe 2026-02-25T15:02:10.604 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.604 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.604 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.604 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.604 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.604 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.604 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.604 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_misc_pp 2026-02-25T15:02:10.604 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49292 2026-02-25T15:02:10.604 INFO:tasks.workunit.client.0.trial189.stdout:test api_misc_pp on pid 49292 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_misc_pp on pid 49292' 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49292 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_pool 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_pool' 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.608 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_pool 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_asio.log 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.609 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_asio: /' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49209/exe ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49209/exe 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.610 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.611 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.611 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.611 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.611 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.611 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.611 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.611 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.611 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.611 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.611 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.611 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.611 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_pool 2026-02-25T15:02:10.611 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49312 2026-02-25T15:02:10.611 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_pool on pid 49312' 2026-02-25T15:02:10.611 INFO:tasks.workunit.client.0.trial189.stdout:test api_pool on pid 49312 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49312 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_snapshots 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.613 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_snapshots' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_list.log 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49230/exe ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_list: /' 2026-02-25T15:02:10.614 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_snapshots 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49230/exe 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.615 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.616 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.616 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.616 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.616 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.616 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.616 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.616 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.616 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.616 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_snapshots 2026-02-25T15:02:10.616 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49331 2026-02-25T15:02:10.616 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_snapshots on pid 49331' 2026-02-25T15:02:10.616 INFO:tasks.workunit.client.0.trial189.stdout:test api_snapshots on pid 49331 2026-02-25T15:02:10.618 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49331 2026-02-25T15:02:10.618 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.618 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.618 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2026-02-25T15:02:10.618 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.618 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.618 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.618 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.618 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.618 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.618 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.618 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_snapshots_pp 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_snapshots_pp' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_lock.log 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_lock: /' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_snapshots_pp 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.619 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49251/exe ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.620 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49251/exe 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_snapshots_pp 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49351 2026-02-25T15:02:10.621 INFO:tasks.workunit.client.0.trial189.stdout:test api_snapshots_pp on pid 49351 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_snapshots_pp on pid 49351' 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49351 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_stat 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_stat' 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.623 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49270/exe ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_stat 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_lock_pp.log 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_lock_pp: /' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.624 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49270/exe 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_stat 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49384 2026-02-25T15:02:10.625 INFO:tasks.workunit.client.0.trial189.stdout:test api_stat on pid 49384 2026-02-25T15:02:10.627 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_stat on pid 49384' 2026-02-25T15:02:10.627 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49384 2026-02-25T15:02:10.627 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.627 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.627 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.627 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.627 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_stat_pp 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_stat_pp' 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_misc.log 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_stat_pp 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_misc: /' 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.628 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.629 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49292/exe ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49292/exe 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_stat_pp 2026-02-25T15:02:10.630 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49413 2026-02-25T15:02:10.631 INFO:tasks.workunit.client.0.trial189.stdout:test api_stat_pp on pid 49413 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_stat_pp on pid 49413' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49413 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_watch_notify 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_watch_notify' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_watch_notify 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.633 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49312/exe ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.634 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49312/exe 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_misc_pp.log 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_misc_pp: /' 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_watch_notify 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49448 2026-02-25T15:02:10.635 INFO:tasks.workunit.client.0.trial189.stdout:test api_watch_notify on pid 49448 2026-02-25T15:02:10.637 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_watch_notify on pid 49448' 2026-02-25T15:02:10.637 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49448 2026-02-25T15:02:10.637 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.637 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.637 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_watch_notify_pp 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_watch_notify_pp' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.638 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49331/exe ']' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_watch_notify_pp 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_pool.log 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.639 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_pool: /' 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49331/exe 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.640 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_watch_notify_pp 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49468 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_watch_notify_pp on pid 49468' 2026-02-25T15:02:10.641 INFO:tasks.workunit.client.0.trial189.stdout:test api_watch_notify_pp on pid 49468 2026-02-25T15:02:10.646 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49468 2026-02-25T15:02:10.646 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.646 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.646 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.646 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.646 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_cmd 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_cmd' 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_snapshots.log 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.647 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_snapshots: /' 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.648 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_cmd 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49351/exe ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.649 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49351/exe 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_cmd 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.650 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49500 2026-02-25T15:02:10.651 INFO:tasks.workunit.client.0.trial189.stdout:test api_cmd on pid 49500 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_cmd on pid 49500' 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49500 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_cmd_pp 2026-02-25T15:02:10.653 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_snapshots_pp: /' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_cmd_pp' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_cmd_pp 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49384/exe ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.654 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49384/exe 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_cmd_pp 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.655 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.656 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.656 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49525 2026-02-25T15:02:10.656 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_cmd_pp on pid 49525' 2026-02-25T15:02:10.656 INFO:tasks.workunit.client.0.trial189.stdout:test api_cmd_pp on pid 49525 2026-02-25T15:02:10.657 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49525 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_service 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_service' 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_service 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_stat.log 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.658 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49413/exe ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_stat: /' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49413/exe 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.659 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_service 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49577 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_service on pid 49577' 2026-02-25T15:02:10.660 INFO:tasks.workunit.client.0.trial189.stdout:test api_service on pid 49577 2026-02-25T15:02:10.662 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49577 2026-02-25T15:02:10.662 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.662 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.662 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2026-02-25T15:02:10.662 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.662 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49448/exe ']' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_service_pp 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_service_pp' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49448/exe 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_service_pp 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_stat_pp.log 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_stat_pp: /' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.663 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.664 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_service_pp 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_watch_notify.log 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49606 2026-02-25T15:02:10.665 INFO:tasks.workunit.client.0.trial189.stdout:test api_service_pp on pid 49606 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_service_pp on pid 49606' 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49606 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_watch_notify: /' 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_c_write_operations 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_c_write_operations' 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_c_write_operations 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.667 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.668 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49468/exe ']' 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49468/exe 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_c_write_operations 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49628 2026-02-25T15:02:10.669 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_c_write_operations on pid 49628' 2026-02-25T15:02:10.670 INFO:tasks.workunit.client.0.trial189.stdout:test api_c_write_operations on pid 49628 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49628 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s api_c_read_operations 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' api_c_read_operations' 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.672 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ echo api_c_read_operations 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.673 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.674 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.675 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=api_c_read_operations 2026-02-25T15:02:10.675 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.675 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.675 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.675 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.675 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.675 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.675 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49500/exe ']' 2026-02-25T15:02:10.675 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49663 2026-02-25T15:02:10.675 INFO:tasks.workunit.client.0.trial189.stdout:test api_c_read_operations on pid 49663 2026-02-25T15:02:10.677 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test api_c_read_operations on pid 49663' 2026-02-25T15:02:10.677 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49663 2026-02-25T15:02:10.677 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s list_parallel 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49500/exe 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' list_parallel' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ echo list_parallel 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.678 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.679 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49525/exe ']' 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49525/exe 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_cmd.log 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_cmd: /' 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.680 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=list_parallel 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49577/exe ']' 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49705 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test list_parallel on pid 49705' 2026-02-25T15:02:10.681 INFO:tasks.workunit.client.0.trial189.stdout:test list_parallel on pid 49705 2026-02-25T15:02:10.685 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49705 2026-02-25T15:02:10.685 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.685 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.685 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.685 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2026-02-25T15:02:10.685 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s open_pools_parallel 2026-02-25T15:02:10.685 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' open_pools_parallel' 2026-02-25T15:02:10.685 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49577/exe 2026-02-25T15:02:10.685 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.685 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.685 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.685 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ echo open_pools_parallel 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.686 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.687 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_cmd_pp: /' 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.688 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.689 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.690 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.690 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.690 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.690 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.690 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.690 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.690 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2026-02-25T15:02:10.690 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_service.log 2026-02-25T15:02:10.690 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_service: /' 2026-02-25T15:02:10.690 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=open_pools_parallel 2026-02-25T15:02:10.690 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49727 2026-02-25T15:02:10.690 INFO:tasks.workunit.client.0.trial189.stdout:test open_pools_parallel on pid 49727 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test open_pools_parallel on pid 49727' 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49727 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s delete_pools_parallel 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' delete_pools_parallel' 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.693 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49606/exe ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49606/exe 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.694 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ echo delete_pools_parallel 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.695 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_service_pp.log 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49628/exe ']' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_service_pp: /' 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=delete_pools_parallel 2026-02-25T15:02:10.696 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49773 2026-02-25T15:02:10.697 INFO:tasks.workunit.client.0.trial189.stdout:test delete_pools_parallel on pid 49773 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test delete_pools_parallel on pid 49773' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49773 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49628/exe 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s cls 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.699 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' cls' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ echo cls 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_c_write_operations: /' 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.700 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49663/exe ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.701 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49663/exe 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=cls 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49794 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test cls on pid 49794' 2026-02-25T15:02:10.702 INFO:tasks.workunit.client.0.trial189.stdout:test cls on pid 49794 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49794 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s cmd 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' cmd' 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.704 INFO:tasks.workunit.client.0.trial189.stderr:++ echo cmd 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.705 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ api_c_read_operations: /' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49705/exe ']' 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=cmd 2026-02-25T15:02:10.706 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49819 2026-02-25T15:02:10.707 INFO:tasks.workunit.client.0.trial189.stdout:test cmd on pid 49819 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test cmd on pid 49819' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49819 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49705/exe 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s handler_error 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' handler_error' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:++ echo handler_error 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.709 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49727/exe ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_list_parallel.log 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ list_parallel: /' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49727/exe 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.710 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=handler_error 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49853 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test handler_error on pid 49853' 2026-02-25T15:02:10.711 INFO:tasks.workunit.client.0.trial189.stdout:test handler_error on pid 49853 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49853 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s io 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' io' 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:++ echo io 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.713 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.714 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.715 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.715 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.715 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2026-02-25T15:02:10.715 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2026-02-25T15:02:10.715 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ open_pools_parallel: /' 2026-02-25T15:02:10.715 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=io 2026-02-25T15:02:10.715 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49877 2026-02-25T15:02:10.715 INFO:tasks.workunit.client.0.trial189.stdout:test io on pid 49877 2026-02-25T15:02:10.716 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test io on pid 49877' 2026-02-25T15:02:10.716 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49877 2026-02-25T15:02:10.716 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2026-02-25T15:02:10.716 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s ec_io 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' ec_io' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ echo ec_io 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49773/exe ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49773/exe 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.717 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.718 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.719 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.719 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.719 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.719 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.719 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.719 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=ec_io 2026-02-25T15:02:10.719 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49914 2026-02-25T15:02:10.719 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test ec_io on pid 49914' 2026-02-25T15:02:10.719 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2026-02-25T15:02:10.719 INFO:tasks.workunit.client.0.trial189.stdout:test ec_io on pid 49914 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49914 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s list 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' list' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:++ echo list 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ delete_pools_parallel: /' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.721 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.722 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49794/exe ']' 2026-02-25T15:02:10.723 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49794/exe 2026-02-25T15:02:10.724 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=list 2026-02-25T15:02:10.724 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.724 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.724 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.724 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.724 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.724 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.724 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49948 2026-02-25T15:02:10.724 INFO:tasks.workunit.client.0.trial189.stdout:test list on pid 49948 2026-02-25T15:02:10.728 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test list on pid 49948' 2026-02-25T15:02:10.728 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49948 2026-02-25T15:02:10.728 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2026-02-25T15:02:10.728 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.728 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2026-02-25T15:02:10.728 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.728 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s ec_list 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' ec_list' 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.729 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:++ echo ec_list 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.730 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.731 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_neorados_cls 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49819/exe ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_neorados_cls.log 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ cls: /' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49819/exe 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=ec_list 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.732 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49979 2026-02-25T15:02:10.733 INFO:tasks.workunit.client.0.trial189.stdout:test ec_list on pid 49979 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test ec_list on pid 49979' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=49979 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s misc 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' misc' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ echo misc 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.735 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49853/exe ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.736 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_neorados_cmd 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49853/exe 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_neorados_cmd.log 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ cmd: /' 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.737 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=misc 2026-02-25T15:02:10.738 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.738 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=50011 2026-02-25T15:02:10.738 INFO:tasks.workunit.client.0.trial189.stdout:test misc on pid 50011 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test misc on pid 50011' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=50011 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s pool 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' pool' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:++ echo pool 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.740 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_neorados_handler_error 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_neorados_handler_error.log 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.741 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ handler_error: /' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49877/exe ']' 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=pool 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=50038 2026-02-25T15:02:10.742 INFO:tasks.workunit.client.0.trial189.stdout:test pool on pid 50038 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test pool on pid 50038' 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=50038 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49877/exe 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s read_operations 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.745 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' read_operations' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ echo read_operations 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.746 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_neorados_io 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_neorados_io.log 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.747 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ io: /' 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=read_operations 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=50067 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test read_operations on pid 50067' 2026-02-25T15:02:10.748 INFO:tasks.workunit.client.0.trial189.stdout:test read_operations on pid 50067 2026-02-25T15:02:10.750 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.750 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=50067 2026-02-25T15:02:10.750 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.750 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49914/exe ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s snapshots 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49914/exe 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' snapshots' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ echo snapshots 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.751 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49948/exe ']' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49948/exe 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.752 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_neorados_ec_io 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.753 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_neorados_ec_io.log 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ ec_io: /' 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=snapshots 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=50106 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test snapshots on pid 50106' 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.754 INFO:tasks.workunit.client.0.trial189.stdout:test snapshots on pid 50106 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=50106 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s watch_notify 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_neorados_list 2026-02-25T15:02:10.756 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' watch_notify' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_neorados_list.log 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ list: /' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ echo watch_notify 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.757 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/49979/exe ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/49979/exe 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=watch_notify 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=50133 2026-02-25T15:02:10.758 INFO:tasks.workunit.client.0.trial189.stdout:test watch_notify on pid 50133 2026-02-25T15:02:10.760 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test watch_notify on pid 50133' 2026-02-25T15:02:10.760 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=50133 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++ printf %25s write_operations 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+ r=' write_operations' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/50011/exe ']' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++ echo write_operations 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++ awk '{print $1}' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.761 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_neorados_ec_list 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/50011/exe 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_neorados_ec_list.log 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ ec_list: /' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.762 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.763 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.764 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.764 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.764 INFO:tasks.workunit.client.0.trial189.stderr:+ ff=write_operations 2026-02-25T15:02:10.764 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=50168 2026-02-25T15:02:10.764 INFO:tasks.workunit.client.0.trial189.stdout:test write_operations on pid 50168 2026-02-25T15:02:10.766 INFO:tasks.workunit.client.0.trial189.stderr:+ echo 'test write_operations on pid 50168' 2026-02-25T15:02:10.766 INFO:tasks.workunit.client.0.trial189.stderr:+ pids[$f]=50168 2026-02-25T15:02:10.766 INFO:tasks.workunit.client.0.trial189.stderr:+ ret=0 2026-02-25T15:02:10.766 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' 1 -eq 1 ']' 2026-02-25T15:02:10.766 INFO:tasks.workunit.client.0.trial189.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2026-02-25T15:02:10.766 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:02:10.766 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49193 2026-02-25T15:02:10.766 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49193 2026-02-25T15:02:10.766 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.767 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.767 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.767 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.767 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.767 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.767 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.767 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.767 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.767 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.767 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.767 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.768 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.768 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.768 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.768 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.768 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.768 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.768 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_neorados_misc 2026-02-25T15:02:10.768 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.768 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.768 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.768 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.768 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.768 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_neorados_misc.log 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [==========] Running 29 tests from 1 test suite. 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [----------] Global test environment set-up. 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [----------] 29 tests from AsioRados 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 10 monclient: get_monmap_and_config 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 10 monclient: build_initial_monmap 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 10 monclient: monmap: 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: epoch 0 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: last_changed 0.000000 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: created 0.000000 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: min_mon_release 0 (unknown) 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: election_strategy: 1 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 1 Processor -- start 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 1 -- start start 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 10 monclient: init 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 10 monclient: _reopen_session rank -1 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 10 monclient(hunting): picked mon.noname-b con 0x55659da75ba0 addr v1:10.20.193.93:6789/0 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 10 monclient(hunting): picked mon.noname-c con 0x55659da98b20 addr v1:10.20.193.94:6789/0 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 10 monclient(hunting): picked mon.noname-a con 0x55659daba6b0 addr v1:10.20.193.90:6789/0 2026-02-25T15:02:10.769 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55659da5bd10 con 0x55659daba6b0 2026-02-25T15:02:10.770 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55659da5cf10 con 0x55659da75ba0 2026-02-25T15:02:10.770 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55659da5e110 con 0x55659da98b20 2026-02-25T15:02:10.770 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 10 monclient(hunting): _renew_subs 2026-02-25T15:02:10.770 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.606+0000 7f0d88111880 10 monclient(hunting): authenticate will time out at 3038.304592s 2026-02-25T15:02:10.770 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.619+0000 7f0d863ca640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x55659daba6b0 0x55659da5a410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.189:49484/0 (socket says 10.20.193.189:49484) 2026-02-25T15:02:10.770 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.619+0000 7f0d863ca640 1 -- 10.20.193.189:0/148164682 learned_addr learned my addr 10.20.193.189:0/148164682 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:02:10.770 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.637+0000 7f0d77fff640 1 -- 10.20.193.189:0/148164682 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3864568067 0 0) 0x55659da5bd10 con 0x55659daba6b0 2026-02-25T15:02:10.770 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.637+0000 7f0d77fff640 10 monclient(hunting): _negotiate 2026-02-25T15:02:10.770 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.637+0000 7f0d77fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:02:10.770 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.637+0000 7f0d77fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:02:10.770 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.637+0000 7f0d77fff640 10 monclient(hunting): my global_id is 4737 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ misc: /' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -f /etc/bashrc ']' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+ . /etc/bashrc 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -z '' ']' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:++ BASHRCSOURCED=Y 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:++ shopt -q login_shell 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+++ umask 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' 0022 -eq 0 ']' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.777 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ SHELL=/bin/bash 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/bash_completion.sh 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorgrep.sh 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.778 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/50038/exe ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/50038/exe 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/50067/exe ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.779 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/50067/exe 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'grep=grep --color=auto' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'egrep=egrep --color=auto' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'fgrep=fgrep --color=auto' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorls.sh 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' '!' -t 0 ']' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ return 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorsysstat.sh 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ export S_COLORS=auto 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ S_COLORS=auto 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorxzgrep.sh 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.780 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_neorados_pool 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_neorados_pool.log 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ pool: /' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_neorados_read_operations 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_neorados_read_operations.log 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ read_operations: /' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.781 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/colorzgrep.sh 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ /usr/libexec/grepconf.sh -c 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/50106/exe ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/50106/exe 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.782 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zgrep=zgrep --color=auto' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ alias 'zegrep=zegrep --color=auto' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/debuginfod.sh 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ prefix=/usr 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z /etc/keys/ima: ']' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset prefix 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/gawk.sh 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/lang.sh 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_neorados_snapshots 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_neorados_snapshots.log 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/locale 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ snapshots: /' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.783 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/50133/exe ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG_backup=en_US.UTF-8 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /etc/locale.conf ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -x /usr/bin/sed ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/50133/exe 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ eval export LANG=en_US.UTF-8 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++++ export LANG=en_US.UTF-8 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++++ LANG=en_US.UTF-8 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ LANG=en_US.UTF-8 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ unset LANG_backup config 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n '' ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -n en_US.UTF-8 ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' dumb = linux ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/less.sh 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++ for i in /etc/profile.d/*.sh 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++ '[' '' ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++ . /etc/profile.d/which2.sh 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ '[' -r /proc/50168/exe ']' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.784 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+++++ readlink /proc/50168/exe 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_neorados_watch_notify 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_neorados_watch_notify.log 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ watch_notify: /' 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:++++ basename /usr/bin/bash 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+++ SHELLNAME=bash 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+++ case "$SHELLNAME" in 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_declare='declare -f' 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+++ which_opt=-f 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+++ export which_declare 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+++ export -f which 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:++ unset i 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:++ unset -f pathmunge 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+ export PATH 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+ unset rc 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+ ceph_test_neorados_write_operations 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+ tee ceph_test_neorados_write_operations.log 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stderr:+ sed 's/^/ write_operations: /' 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.637+0000 7f0d77fff640 1 -- 10.20.193.189:0/148164682 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d64001e10 con 0x55659daba6b0 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.649+0000 7f0d77fff640 1 -- 10.20.193.189:0/148164682 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2909837701 0 0) 0x55659da5cf10 con 0x55659da75ba0 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.649+0000 7f0d77fff640 10 monclient(hunting): _negotiate 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.649+0000 7f0d77fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.649+0000 7f0d77fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.649+0000 7f0d77fff640 10 monclient(hunting): my global_id is 4720 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.649+0000 7f0d77fff640 1 -- 10.20.193.189:0/148164682 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x55659da5bd10 con 0x55659da75ba0 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.663+0000 7f0d77fff640 1 -- 10.20.193.189:0/148164682 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 684904853 0 0) 0x55659da5e110 con 0x55659da98b20 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.663+0000 7f0d77fff640 10 monclient(hunting): _negotiate 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.663+0000 7f0d77fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.663+0000 7f0d77fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.663+0000 7f0d77fff640 10 monclient(hunting): my global_id is 4742 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.663+0000 7f0d77fff640 1 -- 10.20.193.189:0/148164682 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x55659da5cf10 con 0x55659da98b20 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d77fff640 1 -- 10.20.193.189:0/148164682 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 942251593 0 0) 0x55659da5bd10 con 0x55659da75ba0 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d77fff640 1 -- 10.20.193.189:0/148164682 >> v1:10.20.193.94:6789/0 conn(0x55659da98b20 legacy=0x55659dab8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d77fff640 1 -- 10.20.193.189:0/148164682 >> v1:10.20.193.90:6789/0 conn(0x55659daba6b0 legacy=0x55659da5a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:02:10.785 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d77fff640 10 monclient: _finish_hunting 0 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d77fff640 1 monclient: found mon.noname-b 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d77fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d77fff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d77fff640 1 -- 10.20.193.189:0/148164682 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55659da5f3b0 con 0x55659da75ba0 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d77fff640 10 monclient: _finish_auth 0 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d77fff640 10 monclient: _check_auth_tickets 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d77fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d77fff640 1 -- 10.20.193.189:0/148164682 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0d7c001640 con 0x55659da75ba0 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d77fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d77fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d77fff640 10 monclient: dump: 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: epoch 1 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: min_mon_release 20 (tentacle) 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: election_strategy: 3 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d88111880 5 monclient: authenticate success, global_id 4720 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.682+0000 7f0d88111880 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.708+0000 7f0d77fff640 1 -- 10.20.193.189:0/148164682 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0d7c0018e0 con 0x55659da75ba0 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.708+0000 7f0d77fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.708+0000 7f0d77fff640 1 -- 10.20.193.189:0/148164682 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0d7c001cb0 con 0x55659da75ba0 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.708+0000 7f0d77fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.708+0000 7f0d77fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.708+0000 7f0d77fff640 10 monclient: dump: 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: epoch 1 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: min_mon_release 20 (tentacle) 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: election_strategy: 3 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.708+0000 7f0d88111880 10 monclient: get_monmap_and_config success 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.708+0000 7f0d88111880 4 set_mon_vals no callback set 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 10 monclient: shutdown 2026-02-25T15:02:10.786 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:02:10.787 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 1 -- 10.20.193.189:0/148164682 >> v1:10.20.193.93:6789/0 conn(0x55659da75ba0 legacy=0x55659dadab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:02:10.787 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 1 -- 10.20.193.189:0/148164682 shutdown_connections 2026-02-25T15:02:10.787 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 1 -- 10.20.193.189:0/148164682 >> 10.20.193.189:0/148164682 conn(0x55659d9c7970 msgr2=0x55659da76ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:02:10.787 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 1 -- 10.20.193.189:0/148164682 shutdown_connections 2026-02-25T15:02:10.787 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 1 -- 10.20.193.189:0/148164682 wait complete. 2026-02-25T15:02:10.787 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 10 monclient: build_initial_monmap 2026-02-25T15:02:10.787 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 10 monclient: monmap: 2026-02-25T15:02:10.787 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: epoch 0 2026-02-25T15:02:10.787 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:02:10.787 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: last_changed 0.000000 2026-02-25T15:02:10.787 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: created 0.000000 2026-02-25T15:02:10.787 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: min_mon_release 0 (unknown) 2026-02-25T15:02:10.787 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: election_strategy: 1 2026-02-25T15:02:10.787 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:02:10.794 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:02:10.794 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 1 librados: starting msgr at 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 1 librados: starting objecter 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 1 Processor -- start 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 1 -- start start 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 1 librados: setting wanted keys 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 1 librados: calling monclient init 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 10 monclient: init 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 10 monclient: _reopen_session rank -1 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 10 monclient(hunting): picked mon.noname-a con 0x55659da75ba0 addr v1:10.20.193.90:6789/0 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 10 monclient(hunting): picked mon.noname-c con 0x55659da98b20 addr v1:10.20.193.94:6789/0 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 10 monclient(hunting): picked mon.noname-b con 0x55659daba6b0 addr v1:10.20.193.93:6789/0 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55659da63680 con 0x55659da75ba0 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55659da64880 con 0x55659daba6b0 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55659da65a80 con 0x55659da98b20 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 10 monclient(hunting): _renew_subs 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.709+0000 7f0d88111880 10 monclient(hunting): authenticate will time out at 3038.407934s 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.733+0000 7f0d853c8640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x55659da98b20 0x55659d9ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.189:43118/0 (socket says 10.20.193.189:43118) 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.733+0000 7f0d853c8640 1 -- 10.20.193.189:0/3178462082 learned_addr learned my addr 10.20.193.189:0/3178462082 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.733+0000 7f0d76ffd640 10 client.?.objecter ms_handle_connect 0x55659da98b20 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.733+0000 7f0d76ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.758+0000 7f0d76ffd640 10 client.?.objecter ms_handle_connect 0x55659da75ba0 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.758+0000 7f0d76ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.766+0000 7f0d76ffd640 10 client.?.objecter ms_handle_connect 0x55659daba6b0 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.766+0000 7f0d76ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.793+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1276200831 0 0) 0x55659da65a80 con 0x55659da98b20 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.794+0000 7f0d76ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:02:10.795 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.794+0000 7f0d76ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.794+0000 7f0d76ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.794+0000 7f0d76ffd640 10 monclient(hunting): my global_id is 4859 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.794+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d64001a50 con 0x55659da98b20 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.794+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 338805853 0 0) 0x55659da63680 con 0x55659da75ba0 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.794+0000 7f0d76ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.794+0000 7f0d76ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.794+0000 7f0d76ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.794+0000 7f0d76ffd640 10 monclient(hunting): my global_id is 4863 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.794+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x55659da65a80 con 0x55659da75ba0 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.811+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3000027983 0 0) 0x7f0d64001a50 con 0x55659da98b20 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.811+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x55659da63680 con 0x55659da98b20 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.811+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0d700019d0 con 0x55659da98b20 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.811+0000 7f0d76ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.811+0000 7f0d76ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.811+0000 7f0d76ffd640 10 monclient(hunting): dump: 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: epoch 1 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:02:10.854 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:02:10.855 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:02:10.855 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: min_mon_release 20 (tentacle) 2026-02-25T15:02:10.855 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: election_strategy: 3 2026-02-25T15:02:10.855 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:02:10.855 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:02:10.855 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:02:10.855 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.855 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.811+0000 7f0d76ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:02:10.855 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.813+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2020428868 0 0) 0x55659da64880 con 0x55659daba6b0 2026-02-25T15:02:10.855 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.813+0000 7f0d76ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:02:10.855 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.813+0000 7f0d76ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:02:10.855 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.813+0000 7f0d76ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:02:10.855 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.813+0000 7f0d76ffd640 10 monclient(hunting): my global_id is 4837 2026-02-25T15:02:10.855 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.813+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d64001a50 con 0x55659daba6b0 2026-02-25T15:02:11.091 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3125268460 0 0) 0x55659da63680 con 0x55659da98b20 2026-02-25T15:02:11.092 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 >> v1:10.20.193.93:6789/0 conn(0x55659daba6b0 legacy=0x55659dab9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:02:11.092 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 >> v1:10.20.193.90:6789/0 conn(0x55659da75ba0 legacy=0x55659dadab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:02:11.092 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d76ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:02:11.092 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d76ffd640 1 monclient: found mon.c 2026-02-25T15:02:11.092 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d76ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:02:11.092 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d76ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:02:11.092 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55659da66d20 con 0x55659da98b20 2026-02-25T15:02:11.092 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d76ffd640 10 monclient: _finish_auth 0 2026-02-25T15:02:11.092 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d76ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:02:11.092 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:02:11.092 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d88111880 5 monclient: authenticate success, global_id 4859 2026-02-25T15:02:11.093 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d88111880 10 monclient: _renew_subs 2026-02-25T15:02:11.093 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d88111880 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:02:11.093 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x55659da638b0 con 0x55659da98b20 2026-02-25T15:02:11.093 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d88111880 10 client.4859.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:02:11.093 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d88111880 10 monclient: _renew_subs 2026-02-25T15:02:11.093 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d88111880 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:02:11.093 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x55659da63d80 con 0x55659da98b20 2026-02-25T15:02:11.093 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d88111880 1 librados: init done 2026-02-25T15:02:11.093 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:10.853+0000 7f0d88111880 10 librados: wait_for_osdmap waiting 2026-02-25T15:02:11.093 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.090+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0d70002250 con 0x55659da98b20 2026-02-25T15:02:11.093 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.090+0000 7f0d76ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:02:11.093 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.090+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0d70002620 con 0x55659da98b20 2026-02-25T15:02:11.094 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.090+0000 7f0d76ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:02:11.094 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.090+0000 7f0d76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:02:11.094 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.090+0000 7f0d76ffd640 10 monclient: dump: 2026-02-25T15:02:11.094 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: epoch 1 2026-02-25T15:02:11.094 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:02:11.094 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:02:11.094 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:02:11.094 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: min_mon_release 20 (tentacle) 2026-02-25T15:02:11.094 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: election_strategy: 3 2026-02-25T15:02:11.688 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:02:11.688 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:02:11.689 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:02:11.689 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.689 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.090+0000 7f0d77fff640 4 set_mon_vals no callback set 2026-02-25T15:02:11.689 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.091+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f0d70002830 con 0x55659da98b20 2026-02-25T15:02:11.689 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.092+0000 7f0d85bc9640 10 monclient: get_auth_request con 0x7f0d64078dc0 auth_method 0 2026-02-25T15:02:11.689 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.092+0000 7f0d76ffd640 10 client.4859.objecter ms_handle_connect 0x7f0d64078dc0 2026-02-25T15:02:11.689 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.095+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(29..29 src has 1..29) ==== 8141+0+0 (unknown 2043172091 0 0) 0x7f0d700945d0 con 0x55659da98b20 2026-02-25T15:02:11.689 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.095+0000 7f0d76ffd640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_map(29..29 src has 1..29) 2026-02-25T15:02:11.689 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.095+0000 7f0d76ffd640 3 client.4859.objecter handle_osd_map got epochs [29,29] > 0 2026-02-25T15:02:11.689 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.095+0000 7f0d76ffd640 3 client.4859.objecter handle_osd_map decoding full epoch 29 2026-02-25T15:02:11.689 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.095+0000 7f0d76ffd640 20 client.4859.objecter dump_active .. 0 homeless 2026-02-25T15:02:11.689 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.095+0000 7f0d88111880 10 librados: wait_for_osdmap done waiting 2026-02-25T15:02:11.690 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.095+0000 7f0d88111880 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:02:11.690 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.095+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x55659d975da0 con 0x55659da98b20 2026-02-25T15:02:11.690 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.102+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (unknown 1488460068 0 0) 0x7f0d70059ba0 con 0x55659da98b20 2026-02-25T15:02:11.690 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.102+0000 7f0d76ffd640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-25T15:02:11.690 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.102+0000 7f0d777fe640 10 client.4859.objecter _get_latest_version latest 29, have it 2026-02-25T15:02:11.690 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.102+0000 7f0d88111880 10 client.4859.objecter create_pool name=ceph_test_rados_api_asio 2026-02-25T15:02:11.690 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.102+0000 7f0d88111880 10 client.4859.objecter pool_op_submit 1 2026-02-25T15:02:11.690 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.102+0000 7f0d88111880 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:02:11.690 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.102+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6789/0 -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) -- 0x55659d8fa570 con 0x55659da98b20 2026-02-25T15:02:11.690 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.687+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 <== mon.2 v1:10.20.193.94:6789/0 10 ==== pool_op_reply(tid 1 (0) Success v30) ==== 43+0+0 (unknown 2713791762 0 0) 0x7f0d7005d830 con 0x55659da98b20 2026-02-25T15:02:11.690 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.687+0000 7f0d76ffd640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 pool_op_reply(tid 1 (0) Success v30) 2026-02-25T15:02:11.690 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.687+0000 7f0d76ffd640 10 client.4859.objecter handle_pool_op_reply pool_op_reply(tid 1 (0) Success v30) 2026-02-25T15:02:11.691 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.687+0000 7f0d76ffd640 10 client.4859.objecter have request 1 at 0x55659d9cb400 Op: create 2026-02-25T15:02:11.691 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.687+0000 7f0d76ffd640 20 client.4859.objecter waiting for client to reach epoch 30 before calling back 2026-02-25T15:02:14.710 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.687+0000 7f0d76ffd640 10 client.4859.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.687+0000 7f0d76ffd640 10 monclient: _renew_subs 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.687+0000 7f0d76ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.687+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=30}) -- 0x55659da63680 con 0x55659da98b20 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.687+0000 7f0d76ffd640 10 client.4859.objecter done 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 <== mon.2 v1:10.20.193.94:6789/0 11 ==== osd_map(30..30 src has 1..30) ==== 13374+0+0 (unknown 3669251848 0 0) 0x7f0d70062bb0 con 0x55659da98b20 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d76ffd640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_map(30..30 src has 1..30) 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d76ffd640 3 client.4859.objecter handle_osd_map got epochs [30,30] > 29 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d76ffd640 3 client.4859.objecter handle_osd_map decoding incremental epoch 30 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d76ffd640 20 client.4859.objecter dump_active .. 0 homeless 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d88111880 10 librados: set snap read head -> 1 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d88111880 10 librados: writefull oid=exist nspace= 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da640c0 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 30 base exist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [writefull 0~5 in=5b] tid 2 osd.10 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 2 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d88111880 15 client.4859.objecter _send_op 2 to 25.1 on osd.10 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:2 25.1 25:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55659da66570 con 0x55659da66160 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.693+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.880+0000 7f0d863ca640 10 monclient: get_auth_request con 0x55659da66160 auth_method 0 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:11.902+0000 7f0d76ffd640 10 client.4859.objecter ms_handle_connect 0x55659da66160 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:13.709+0000 7f0d75ffb640 10 monclient: tick 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:13.710+0000 7f0d75ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:13.710+0000 7f0d75ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:02:14.711 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:13.710+0000 7f0d75ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:02:16.344 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:14.710+0000 7f0d75ffb640 10 monclient: tick 2026-02-25T15:02:16.344 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:14.710+0000 7f0d75ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:02:16.344 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:14.710+0000 7f0d75ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:02:16.344 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.710+0000 7f0d75ffb640 10 monclient: tick 2026-02-25T15:02:16.344 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.710+0000 7f0d75ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:02:16.344 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.710+0000 7f0d75ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:02:16.344 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.854+0000 7f0d87e54640 10 client.4859.objecter tick 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.854+0000 7f0d87e54640 20 client.4859.objecter tick: elapsed since last osdmap request 4.16699s, stale_window 10s, epoch 31, monc->get_start(osdmap) 0 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 1 ==== osd_map(31..33 src has 1..33) ==== 13897+0+0 (unknown 414529525 0 0) 0x7f0d78001830 con 0x55659da66160 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_map(31..33 src has 1..33) 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 3 client.4859.objecter handle_osd_map got epochs [31,33] > 30 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 3 client.4859.objecter handle_osd_map decoding incremental epoch 31 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 10 client.4859.objecter checking op 2 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 20 client.4859.objecter _calc_target epoch 31 base exist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 3 client.4859.objecter handle_osd_map decoding incremental epoch 32 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 10 client.4859.objecter checking op 2 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 20 client.4859.objecter _calc_target epoch 32 base exist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 3 client.4859.objecter handle_osd_map decoding incremental epoch 33 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 10 client.4859.objecter checking op 2 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 20 client.4859.objecter dump_active .. 0 homeless 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:15.891+0000 7f0d76ffd640 20 client.4859.objecter 2 25.d3a59201 osd.10 exist [writefull 0~5 in=5b] 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 2 ==== osd_op_reply(2 exist [writefull 0~5] v33'1 uv1 ondisk = 0) ==== 149+0+0 (unknown 2404485114 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(2 exist [writefull 0~5] v33'1 uv1 ondisk = 0) 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.345 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 2 ondisk uv 1 in 25.1 attempt 0 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 2 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d863ca640 15 client.4859.objecter _finish_op 2 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 2 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 10 librados: Objecter returned from writefull r=0 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da640c0 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 10 client.4859.objecter chose random osd.10 of [10,13] 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [read 0~256] tid 3 osd.10 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 3 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 15 client.4859.objecter _send_op 3 to 25.1 on osd.10 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:3 25.1 25:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da64a50 con 0x55659da66160 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da64f10 2026-02-25T15:02:16.346 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.347 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.347 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.347 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 10 client.4859.objecter chose random osd.13 of [10,13] 2026-02-25T15:02:16.347 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.347 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [read 0~256] tid 4 osd.13 2026-02-25T15:02:16.347 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.347 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 13 4 2026-02-25T15:02:16.393 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 15 client.4859.objecter _send_op 4 to 25.1 on osd.13 2026-02-25T15:02:16.393 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.93:6812/1850919728 -- osd_op(unknown.0.0:4 25.1 25:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6a1a0 con 0x55659da68610 2026-02-25T15:02:16.393 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.393 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.343+0000 7f0d88111880 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.345+0000 7f0d85bc9640 10 monclient: get_auth_request con 0x55659da68610 auth_method 0 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.364+0000 7f0d76ffd640 10 client.4859.objecter ms_handle_connect 0x55659da68610 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.365+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 3 ==== osd_op_reply(3 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) ==== 149+0+5 (unknown 3471160006 0 3741568377) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.365+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(3 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.365+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.365+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 3 ondisk uv 1 in 25.1 attempt 0 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.365+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 5 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.365+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 3 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.365+0000 7f0d863ca640 15 client.4859.objecter _finish_op 3 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.365+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.365+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 3 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.365+0000 7f0d863ca640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d85bc9640 1 -- 10.20.193.189:0/3178462082 <== osd.13 v1:10.20.193.93:6812/1850919728 1 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 1497253407 0 0) 0x7f0d7c002c80 con 0x55659da68610 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d85bc9640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d85bc9640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d85bc9640 7 client.4859.objecter handle_osd_op_reply 4 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d85bc9640 10 client.4859.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d85bc9640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d85bc9640 15 client.4859.objecter handle_osd_op_reply completed tid 4 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d85bc9640 15 client.4859.objecter _finish_op 4 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d85bc9640 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.394 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d85bc9640 15 client.4859.objecter _session_op_remove 13 4 2026-02-25T15:02:16.395 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d85bc9640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.405 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (49 ms) 2026-02-25T15:02:16.405 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncReadDeferred 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da640c0 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 10 client.4859.objecter chose random osd.10 of [10,13] 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [read 0~256] tid 5 osd.10 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 5 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 15 client.4859.objecter _send_op 5 to 25.1 on osd.10 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:5 25.1 25:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da64f10 con 0x55659da66160 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6a690 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 10 client.4859.objecter chose random osd.10 of [10,13] 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [read 0~256] tid 6 osd.10 2026-02-25T15:02:16.406 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 6 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 15 client.4859.objecter _send_op 6 to 25.1 on osd.10 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:6 25.1 25:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6abc0 con 0x55659da66160 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.392+0000 7f0d88111880 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 4 ==== osd_op_reply(5 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) ==== 149+0+5 (unknown 3471160006 0 3741568377) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(5 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 5 ondisk uv 1 in 25.1 attempt 0 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 5 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 5 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 15 client.4859.objecter _finish_op 5 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 5 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 5 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 1497253407 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 6 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 10 client.4859.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 6 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 15 client.4859.objecter _finish_op 6 2026-02-25T15:02:16.407 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.408 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 6 2026-02-25T15:02:16.408 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.408 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncReadDeferred (12 ms) 2026-02-25T15:02:16.408 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2026-02-25T15:02:16.408 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da640c0 2026-02-25T15:02:16.408 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.408 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.408 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.408 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 10 client.4859.objecter chose random osd.10 of [10,13] 2026-02-25T15:02:16.408 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [read 0~256] tid 7 osd.10 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 7 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 15 client.4859.objecter _send_op 7 to 25.1 on osd.10 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:7 25.1 25:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6b090 con 0x55659da66160 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6b760 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 10 client.4859.objecter chose random osd.13 of [10,13] 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [read 0~256] tid 8 osd.13 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 13 8 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 15 client.4859.objecter _send_op 8 to 25.1 on osd.13 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.93:6812/1850919728 -- osd_op(unknown.0.0:8 25.1 25:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6bc10 con 0x55659da68610 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.405+0000 7f0d88111880 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.410 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.409+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 6 ==== osd_op_reply(7 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) ==== 149+0+5 (unknown 3471160006 0 3741568377) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.411 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.409+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(7 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) 2026-02-25T15:02:16.411 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.409+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.411 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.409+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 7 ondisk uv 1 in 25.1 attempt 0 2026-02-25T15:02:16.411 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.409+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 5 2026-02-25T15:02:16.411 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.409+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 7 2026-02-25T15:02:16.411 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.409+0000 7f0d863ca640 15 client.4859.objecter _finish_op 7 2026-02-25T15:02:16.411 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.409+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.409+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 7 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.409+0000 7f0d863ca640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d85bc9640 1 -- 10.20.193.189:0/3178462082 <== osd.13 v1:10.20.193.93:6812/1850919728 2 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 1497253407 0 0) 0x7f0d7c002c80 con 0x55659da68610 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d85bc9640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d85bc9640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d85bc9640 7 client.4859.objecter handle_osd_op_reply 8 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d85bc9640 10 client.4859.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d85bc9640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d85bc9640 15 client.4859.objecter handle_osd_op_reply completed tid 8 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d85bc9640 15 client.4859.objecter _finish_op 8 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d85bc9640 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d85bc9640 15 client.4859.objecter _session_op_remove 13 8 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d85bc9640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (14 ms) 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da640c0 2026-02-25T15:02:16.420 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.421 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.421 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.421 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 10 client.4859.objecter chose random osd.13 of [10,13] 2026-02-25T15:02:16.421 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.421 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [read 0~256] tid 9 osd.13 2026-02-25T15:02:16.421 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.421 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 13 9 2026-02-25T15:02:16.421 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 15 client.4859.objecter _send_op 9 to 25.1 on osd.13 2026-02-25T15:02:16.421 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.93:6812/1850919728 -- osd_op(unknown.0.0:9 25.1 25:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6b740 con 0x55659da68610 2026-02-25T15:02:16.421 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.421 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.421 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c1b0 2026-02-25T15:02:16.427 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.427 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 10 client.4859.objecter chose random osd.10 of [10,13] 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [read 0~256] tid 10 osd.10 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 10 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 15 client.4859.objecter _send_op 10 to 25.1 on osd.10 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:10 25.1 25:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6c660 con 0x55659da66160 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.419+0000 7f0d88111880 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.421+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 7 ==== osd_op_reply(10 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 1497253407 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.421+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(10 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.421+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.421+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 10 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.421+0000 7f0d863ca640 10 client.4859.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.421+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.421+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 10 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.421+0000 7f0d863ca640 15 client.4859.objecter _finish_op 10 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.421+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.421+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 10 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.421+0000 7f0d863ca640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 1 -- 10.20.193.189:0/3178462082 <== osd.13 v1:10.20.193.93:6812/1850919728 3 ==== osd_op_reply(9 exist [read 0~256] v0'0 uv0 ondisk = -11 ((11) Resource temporarily unavailable)) ==== 149+0+0 (unknown 2488840873 0 0) 0x7f0d7c002c80 con 0x55659da68610 2026-02-25T15:02:16.428 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(9 exist [read 0~256] v0'0 uv0 ondisk = -11 ((11) Resource temporarily unavailable)) 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 7 client.4859.objecter handle_osd_op_reply 9 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 7 client.4859.objecter got -EAGAIN, resubmitting 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 15 client.4859.objecter _session_op_remove 13 9 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 10 client.4859.objecter _op_submit op 0x55659da640c0 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 10 client.4859.objecter _op_submit oid exist '@25' '@25' [read 0~256] tid 11 osd.10 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 15 client.4859.objecter _session_op_assign 10 11 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 15 client.4859.objecter _send_op 11 to 25.1 on osd.10 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:11 25.1 25:8049a5cb:::exist:head [read 0~256] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33) -- 0x55659da6b740 con 0x55659da66160 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.427+0000 7f0d85bc9640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.431 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 8 ==== osd_op_reply(11 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) ==== 149+0+5 (unknown 2050540943 0 3741568377) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.432 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(11 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) 2026-02-25T15:02:16.432 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.432 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 11 ondisk uv 1 in 25.1 attempt 1 2026-02-25T15:02:16.432 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 5 2026-02-25T15:02:16.432 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 11 2026-02-25T15:02:16.432 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d863ca640 15 client.4859.objecter _finish_op 11 2026-02-25T15:02:16.432 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.432 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 11 2026-02-25T15:02:16.432 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.432 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (10 ms) 2026-02-25T15:02:16.432 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncReadAwaitable 2026-02-25T15:02:16.432 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 10 client.4859.objecter chose random osd.13 of [10,13] 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [read 0~256] tid 12 osd.13 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 13 12 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 15 client.4859.objecter _send_op 12 to 25.1 on osd.13 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.93:6812/1850919728 -- osd_op(unknown.0.0:12 25.1 25:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6ca50 con 0x55659da68610 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6cf30 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 10 client.4859.objecter chose random osd.10 of [10,13] 2026-02-25T15:02:16.433 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [read 0~256] tid 13 osd.10 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 13 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 15 client.4859.objecter _send_op 13 to 25.1 on osd.10 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:13 25.1 25:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6d3e0 con 0x55659da66160 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.430+0000 7f0d88111880 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 1 -- 10.20.193.189:0/3178462082 <== osd.13 v1:10.20.193.93:6812/1850919728 4 ==== osd_op_reply(12 exist [read 0~256] v0'0 uv0 ondisk = -11 ((11) Resource temporarily unavailable)) ==== 149+0+0 (unknown 2488840873 0 0) 0x7f0d7c002c80 con 0x55659da68610 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(12 exist [read 0~256] v0'0 uv0 ondisk = -11 ((11) Resource temporarily unavailable)) 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 7 client.4859.objecter handle_osd_op_reply 12 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 7 client.4859.objecter got -EAGAIN, resubmitting 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 15 client.4859.objecter _session_op_remove 13 12 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.434 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 10 client.4859.objecter _op_submit oid exist '@25' '@25' [read 0~256] tid 14 osd.10 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 20 client.4859.objecter get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 15 client.4859.objecter _session_op_assign 10 14 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 15 client.4859.objecter _send_op 14 to 25.1 on osd.10 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:14 25.1 25:8049a5cb:::exist:head [read 0~256] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33) -- 0x55659da6ca50 con 0x55659da66160 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d85bc9640 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 9 ==== osd_op_reply(13 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 1497253407 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(13 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 13 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d863ca640 10 client.4859.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 13 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d863ca640 15 client.4859.objecter _finish_op 13 2026-02-25T15:02:16.435 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.493 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 13 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.432+0000 7f0d863ca640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 10 ==== osd_op_reply(14 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) ==== 149+0+5 (unknown 2050540943 0 3741568377) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(14 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 14 ondisk uv 1 in 25.1 attempt 1 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 5 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 14 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d863ca640 15 client.4859.objecter _finish_op 14 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 14 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncReadAwaitable (13 ms) 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d88111880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d88111880 20 librados: queue_aio_write 0x55659d920390 completion 0x55659da64450 write_seq 1 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [write 0~5 in=5b] tid 15 osd.10 2026-02-25T15:02:16.494 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.495 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 15 2026-02-25T15:02:16.495 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d88111880 15 client.4859.objecter _send_op 15 to 25.1 on osd.10 2026-02-25T15:02:16.495 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:15 25.1 25:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55659da6ca50 con 0x55659da66160 2026-02-25T15:02:16.495 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.495 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.495 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.443+0000 7f0d88111880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2026-02-25T15:02:16.515 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 11 ==== osd_op_reply(15 exist [write 0~5] v33'2 uv2 ondisk = 0) ==== 149+0+0 (unknown 2414345490 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.515 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(15 exist [write 0~5] v33'2 uv2 ondisk = 0) 2026-02-25T15:02:16.515 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.515 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 15 ondisk uv 2 in 25.1 attempt 0 2026-02-25T15:02:16.515 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.515 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 15 2026-02-25T15:02:16.515 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d863ca640 15 client.4859.objecter _finish_op 15 2026-02-25T15:02:16.515 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.515 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (49 ms) 2026-02-25T15:02:16.515 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteDeferred 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 15 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d863ca640 20 librados: complete_aio_write 0x55659da64450 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d88111880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d88111880 20 librados: queue_aio_write 0x55659d920390 completion 0x55659da6cf30 write_seq 2 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [write 0~5 in=5b] tid 16 osd.10 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 16 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d88111880 15 client.4859.objecter _send_op 16 to 25.1 on osd.10 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:16 25.1 25:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55659da6ca50 con 0x55659da66160 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.493+0000 7f0d88111880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2026-02-25T15:02:16.516 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 12 ==== osd_op_reply(16 exist [write 0~5] v33'3 uv3 ondisk = 0) ==== 149+0+0 (unknown 4217849542 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.542 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(16 exist [write 0~5] v33'3 uv3 ondisk = 0) 2026-02-25T15:02:16.542 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.542 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 16 ondisk uv 3 in 25.1 attempt 0 2026-02-25T15:02:16.542 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.542 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 16 2026-02-25T15:02:16.542 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d863ca640 15 client.4859.objecter _finish_op 16 2026-02-25T15:02:16.542 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.542 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 16 2026-02-25T15:02:16.542 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.542 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d863ca640 20 librados: complete_aio_write 0x55659da6cf30 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncWriteDeferred (21 ms) 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d88111880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d88111880 20 librados: queue_aio_write 0x55659d920390 completion 0x55659da6d080 write_seq 3 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [write 0~5 in=5b] tid 17 osd.10 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 17 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d88111880 15 client.4859.objecter _send_op 17 to 25.1 on osd.10 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:17 25.1 25:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55659da6e1a0 con 0x55659da66160 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.514+0000 7f0d88111880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 13 ==== osd_op_reply(17 exist [write 0~5] v33'4 uv4 ondisk = 0) ==== 149+0+0 (unknown 2970096411 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.543 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(17 exist [write 0~5] v33'4 uv4 ondisk = 0) 2026-02-25T15:02:16.558 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.558 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 17 ondisk uv 4 in 25.1 attempt 0 2026-02-25T15:02:16.558 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.558 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 17 2026-02-25T15:02:16.558 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d863ca640 15 client.4859.objecter _finish_op 17 2026-02-25T15:02:16.558 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.558 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 17 2026-02-25T15:02:16.558 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.558 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d863ca640 20 librados: complete_aio_write 0x55659da6d080 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (26 ms) 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d88111880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d88111880 20 librados: queue_aio_write 0x55659d920390 completion 0x55659da6e600 write_seq 4 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [write 0~5 in=5b] tid 18 osd.10 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 18 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d88111880 15 client.4859.objecter _send_op 18 to 25.1 on osd.10 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:18 25.1 25:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55659dc091f0 con 0x55659da66160 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.541+0000 7f0d88111880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 14 ==== osd_op_reply(18 exist [write 0~5] v33'5 uv5 ondisk = 0) ==== 149+0+0 (unknown 3314067663 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(18 exist [write 0~5] v33'5 uv5 ondisk = 0) 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.559 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 18 ondisk uv 5 in 25.1 attempt 0 2026-02-25T15:02:16.577 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.577 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 18 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d863ca640 15 client.4859.objecter _finish_op 18 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 18 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (16 ms) 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d863ca640 20 librados: complete_aio_write 0x55659da6e600 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteAwaitable 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d88111880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d88111880 20 librados: queue_aio_write 0x55659d920390 completion 0x55659dc09600 write_seq 5 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [write 0~5 in=5b] tid 19 osd.10 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 19 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d88111880 15 client.4859.objecter _send_op 19 to 25.1 on osd.10 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:19 25.1 25:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55659dbc91b0 con 0x55659da66160 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.578 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.557+0000 7f0d88111880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 15 ==== osd_op_reply(19 exist [write 0~5] v33'6 uv6 ondisk = 0) ==== 149+0+0 (unknown 1477010611 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(19 exist [write 0~5] v33'6 uv6 ondisk = 0) 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 19 ondisk uv 6 in 25.1 attempt 0 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 19 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d863ca640 15 client.4859.objecter _finish_op 19 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 19 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d863ca640 20 librados: complete_aio_write 0x55659dc09600 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncWriteAwaitable (19 ms) 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 10 client.4859.objecter chose random osd.10 of [10,13] 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [read 0~0] tid 20 osd.10 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.579 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 20 2026-02-25T15:02:16.580 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 15 client.4859.objecter _send_op 20 to 25.1 on osd.10 2026-02-25T15:02:16.580 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:20 25.1 25:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659dbc91b0 con 0x55659da66160 2026-02-25T15:02:16.580 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.580 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.580 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6d2a0 2026-02-25T15:02:16.580 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.580 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.580 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.580 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 10 client.4859.objecter chose random osd.13 of [10,13] 2026-02-25T15:02:16.580 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.580 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [read 0~0] tid 21 osd.13 2026-02-25T15:02:16.580 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.580 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 13 21 2026-02-25T15:02:16.580 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 15 client.4859.objecter _send_op 21 to 25.1 on osd.13 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.93:6812/1850919728 -- osd_op(unknown.0.0:21 25.1 25:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6d750 con 0x55659da68610 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.577+0000 7f0d88111880 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 16 ==== osd_op_reply(20 exist [read 0~5 out=5b] v0'0 uv6 ondisk = 0) ==== 149+0+5 (unknown 2291560189 0 3741568377) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(20 exist [read 0~5 out=5b] v0'0 uv6 ondisk = 0) 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 20 ondisk uv 6 in 25.1 attempt 0 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 5 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 20 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d863ca640 15 client.4859.objecter _finish_op 20 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 20 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d863ca640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d85bc9640 1 -- 10.20.193.189:0/3178462082 <== osd.13 v1:10.20.193.93:6812/1850919728 5 ==== osd_op_reply(21 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 884373543 0 0) 0x7f0d7c002c80 con 0x55659da68610 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d85bc9640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(21 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d85bc9640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d85bc9640 7 client.4859.objecter handle_osd_op_reply 21 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d85bc9640 10 client.4859.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d85bc9640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.581 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d85bc9640 15 client.4859.objecter handle_osd_op_reply completed tid 21 2026-02-25T15:02:16.582 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d85bc9640 15 client.4859.objecter _finish_op 21 2026-02-25T15:02:16.582 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (3 ms) 2026-02-25T15:02:16.582 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d85bc9640 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.582 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationDeferred 2026-02-25T15:02:16.582 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d85bc9640 15 client.4859.objecter _session_op_remove 13 21 2026-02-25T15:02:16.582 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d85bc9640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.582 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.582 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 10 client.4859.objecter chose random osd.13 of [10,13] 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [read 0~0] tid 22 osd.13 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 13 22 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 15 client.4859.objecter _send_op 22 to 25.1 on osd.13 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.93:6812/1850919728 -- osd_op(unknown.0.0:22 25.1 25:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659dbc91b0 con 0x55659da68610 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6d2a0 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 10 client.4859.objecter chose random osd.13 of [10,13] 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [read 0~0] tid 23 osd.13 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 13 23 2026-02-25T15:02:16.590 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 15 client.4859.objecter _send_op 23 to 25.1 on osd.13 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.93:6812/1850919728 -- osd_op(unknown.0.0:23 25.1 25:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6db40 con 0x55659da68610 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 5 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.580+0000 7f0d88111880 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 1 -- 10.20.193.189:0/3178462082 <== osd.13 v1:10.20.193.93:6812/1850919728 6 ==== osd_op_reply(22 exist [read 0~0] v0'0 uv0 ondisk = -11 ((11) Resource temporarily unavailable)) ==== 149+0+0 (unknown 4191194257 0 0) 0x7f0d7c002c80 con 0x55659da68610 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(22 exist [read 0~0] v0'0 uv0 ondisk = -11 ((11) Resource temporarily unavailable)) 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 7 client.4859.objecter handle_osd_op_reply 22 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 7 client.4859.objecter got -EAGAIN, resubmitting 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 5 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 15 client.4859.objecter _session_op_remove 13 22 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 10 client.4859.objecter _op_submit oid exist '@25' '@25' [read 0~0] tid 24 osd.10 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 15 client.4859.objecter _session_op_assign 10 24 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 15 client.4859.objecter _send_op 24 to 25.1 on osd.10 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:24 25.1 25:8049a5cb:::exist:head [read 0~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33) -- 0x55659da6db40 con 0x55659da66160 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.591 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 1 -- 10.20.193.189:0/3178462082 <== osd.13 v1:10.20.193.93:6812/1850919728 7 ==== osd_op_reply(23 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 884373543 0 0) 0x7f0d7c002c80 con 0x55659da68610 2026-02-25T15:02:16.592 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(23 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.592 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.592 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 7 client.4859.objecter handle_osd_op_reply 23 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.592 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 10 client.4859.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:02:16.592 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.592 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 15 client.4859.objecter handle_osd_op_reply completed tid 23 2026-02-25T15:02:16.592 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 15 client.4859.objecter _finish_op 23 2026-02-25T15:02:16.592 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.592 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 15 client.4859.objecter _session_op_remove 13 23 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.589+0000 7f0d85bc9640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 17 ==== osd_op_reply(24 exist [read 0~5 out=5b] v0'0 uv6 ondisk = 0) ==== 149+0+5 (unknown 1011554740 0 3741568377) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(24 exist [read 0~5 out=5b] v0'0 uv6 ondisk = 0) 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 24 ondisk uv 6 in 25.1 attempt 1 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 5 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 24 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d863ca640 15 client.4859.objecter _finish_op 24 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 24 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationDeferred (13 ms) 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 10 client.4859.objecter chose random osd.10 of [10,13] 2026-02-25T15:02:16.594 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.595 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [read 0~0] tid 25 osd.10 2026-02-25T15:02:16.595 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.595 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 25 2026-02-25T15:02:16.595 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 15 client.4859.objecter _send_op 25 to 25.1 on osd.10 2026-02-25T15:02:16.595 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:25 25.1 25:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6db40 con 0x55659da66160 2026-02-25T15:02:16.595 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.595 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.595 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6d190 2026-02-25T15:02:16.595 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.595 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.600 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.600 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 10 client.4859.objecter chose random osd.10 of [10,13] 2026-02-25T15:02:16.600 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.600 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [read 0~0] tid 26 osd.10 2026-02-25T15:02:16.600 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.600 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 26 2026-02-25T15:02:16.600 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 15 client.4859.objecter _send_op 26 to 25.1 on osd.10 2026-02-25T15:02:16.600 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:26 25.1 25:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659dbc9720 con 0x55659da66160 2026-02-25T15:02:16.600 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.593+0000 7f0d88111880 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 18 ==== osd_op_reply(25 exist [read 0~5 out=5b] v0'0 uv6 ondisk = 0) ==== 149+0+5 (unknown 2291560189 0 3741568377) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(25 exist [read 0~5 out=5b] v0'0 uv6 ondisk = 0) 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 25 ondisk uv 6 in 25.1 attempt 0 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 5 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 25 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 15 client.4859.objecter _finish_op 25 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 25 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 19 ==== osd_op_reply(26 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 884373543 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(26 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 26 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 10 client.4859.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 26 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 15 client.4859.objecter _finish_op 26 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 26 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (6 ms) 2026-02-25T15:02:16.601 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6d190 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 10 client.4859.objecter chose random osd.13 of [10,13] 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [read 0~0] tid 27 osd.13 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 13 27 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 15 client.4859.objecter _send_op 27 to 25.1 on osd.13 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.93:6812/1850919728 -- osd_op(unknown.0.0:27 25.1 25:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6db40 con 0x55659da68610 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659dbe9730 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 10 client.4859.objecter chose random osd.10 of [10,13] 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [read 0~0] tid 28 osd.10 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 28 2026-02-25T15:02:16.602 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 15 client.4859.objecter _send_op 28 to 25.1 on osd.10 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:28 25.1 25:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6c2a0 con 0x55659da66160 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.599+0000 7f0d88111880 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 1 -- 10.20.193.189:0/3178462082 <== osd.13 v1:10.20.193.93:6812/1850919728 8 ==== osd_op_reply(27 exist [read 0~0] v0'0 uv0 ondisk = -11 ((11) Resource temporarily unavailable)) ==== 149+0+0 (unknown 4191194257 0 0) 0x7f0d7c002c80 con 0x55659da68610 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(27 exist [read 0~0] v0'0 uv0 ondisk = -11 ((11) Resource temporarily unavailable)) 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 7 client.4859.objecter handle_osd_op_reply 27 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 7 client.4859.objecter got -EAGAIN, resubmitting 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 15 client.4859.objecter _session_op_remove 13 27 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 10 client.4859.objecter _op_submit op 0x55659da6d190 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 10 client.4859.objecter _op_submit oid exist '@25' '@25' [read 0~0] tid 29 osd.10 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 20 client.4859.objecter get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 15 client.4859.objecter _session_op_assign 10 29 2026-02-25T15:02:16.603 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 15 client.4859.objecter _send_op 29 to 25.1 on osd.10 2026-02-25T15:02:16.604 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:29 25.1 25:8049a5cb:::exist:head [read 0~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33) -- 0x55659da6db40 con 0x55659da66160 2026-02-25T15:02:16.604 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.604 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d85bc9640 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.604 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 20 ==== osd_op_reply(28 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 884373543 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.604 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(28 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.604 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.604 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 28 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.612 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d863ca640 10 client.4859.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:02:16.612 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.612 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 28 2026-02-25T15:02:16.612 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d863ca640 15 client.4859.objecter _finish_op 28 2026-02-25T15:02:16.612 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 28 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.602+0000 7f0d863ca640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.611+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 21 ==== osd_op_reply(29 exist [read 0~5 out=5b] v0'0 uv6 ondisk = 0) ==== 149+0+5 (unknown 1011554740 0 3741568377) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.611+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(29 exist [read 0~5 out=5b] v0'0 uv6 ondisk = 0) 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.611+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.611+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 29 ondisk uv 6 in 25.1 attempt 1 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.611+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 5 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (12 ms) 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.611+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 29 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationAwaitable 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.611+0000 7f0d863ca640 15 client.4859.objecter _finish_op 29 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.611+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.611+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 29 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.611+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 10 client.4859.objecter chose random osd.10 of [10,13] 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [read 0~0] tid 30 osd.10 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 30 2026-02-25T15:02:16.613 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 15 client.4859.objecter _send_op 30 to 25.1 on osd.10 2026-02-25T15:02:16.628 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:30 25.1 25:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6db40 con 0x55659da66160 2026-02-25T15:02:16.628 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.628 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.628 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6d190 2026-02-25T15:02:16.628 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.628 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.628 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.628 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 10 client.4859.objecter chose random osd.13 of [10,13] 2026-02-25T15:02:16.628 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.628 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [read 0~0] tid 31 osd.13 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 13 31 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 15 client.4859.objecter _send_op 31 to 25.1 on osd.13 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.93:6812/1850919728 -- osd_op(unknown.0.0:31 25.1 25:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659dbc9720 con 0x55659da68610 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.612+0000 7f0d88111880 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.615+0000 7f0d85bc9640 1 -- 10.20.193.189:0/3178462082 <== osd.13 v1:10.20.193.93:6812/1850919728 9 ==== osd_op_reply(31 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 884373543 0 0) 0x7f0d7c002c80 con 0x55659da68610 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.615+0000 7f0d85bc9640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(31 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.615+0000 7f0d85bc9640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.615+0000 7f0d85bc9640 7 client.4859.objecter handle_osd_op_reply 31 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.615+0000 7f0d85bc9640 10 client.4859.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.615+0000 7f0d85bc9640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.615+0000 7f0d85bc9640 15 client.4859.objecter handle_osd_op_reply completed tid 31 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.615+0000 7f0d85bc9640 15 client.4859.objecter _finish_op 31 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.615+0000 7f0d85bc9640 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.615+0000 7f0d85bc9640 15 client.4859.objecter _session_op_remove 13 31 2026-02-25T15:02:16.629 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.615+0000 7f0d85bc9640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.678 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 22 ==== osd_op_reply(30 exist [read 0~5 out=5b] v0'0 uv6 ondisk = 0) ==== 149+0+5 (unknown 2291560189 0 3741568377) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.678 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(30 exist [read 0~5 out=5b] v0'0 uv6 ondisk = 0) 2026-02-25T15:02:16.678 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.678 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 30 ondisk uv 6 in 25.1 attempt 0 2026-02-25T15:02:16.678 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 5 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationAwaitable (15 ms) 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 30 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d863ca640 15 client.4859.objecter _finish_op 30 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 30 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d88111880 20 librados: queue_aio_write 0x55659d920390 completion 0x55659da6df30 write_seq 6 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [writefull 0~5 in=5b] tid 32 osd.10 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 32 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d88111880 15 client.4859.objecter _send_op 32 to 25.1 on osd.10 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:32 25.1 25:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55659da6db40 con 0x55659da66160 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.627+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 23 ==== osd_op_reply(32 exist [writefull 0~5] v33'7 uv7 ondisk = 0) ==== 149+0+0 (unknown 2982046195 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(32 exist [writefull 0~5] v33'7 uv7 ondisk = 0) 2026-02-25T15:02:16.679 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 32 ondisk uv 7 in 25.1 attempt 0 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (49 ms) 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 32 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationDeferred 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d863ca640 15 client.4859.objecter _finish_op 32 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 32 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d863ca640 20 librados: complete_aio_write 0x55659da6df30 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d88111880 20 librados: queue_aio_write 0x55659d920390 completion 0x55659dbc95a0 write_seq 7 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [writefull 0~5 in=5b] tid 33 osd.10 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 33 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d88111880 15 client.4859.objecter _send_op 33 to 25.1 on osd.10 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:33 25.1 25:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55659da6db40 con 0x55659da66160 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.677+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.710+0000 7f0d75ffb640 10 monclient: tick 2026-02-25T15:02:16.726 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.710+0000 7f0d75ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:02:16.727 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.710+0000 7f0d75ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:02:16.727 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 24 ==== osd_op_reply(33 exist [writefull 0~5] v33'8 uv8 ondisk = 0) ==== 149+0+0 (unknown 1373665693 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.727 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(33 exist [writefull 0~5] v33'8 uv8 ondisk = 0) 2026-02-25T15:02:16.727 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.727 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 33 ondisk uv 8 in 25.1 attempt 0 2026-02-25T15:02:16.773 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.773 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 33 2026-02-25T15:02:16.773 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d863ca640 15 client.4859.objecter _finish_op 33 2026-02-25T15:02:16.773 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.773 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 33 2026-02-25T15:02:16.773 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.773 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationDeferred (47 ms) 2026-02-25T15:02:16.773 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d863ca640 20 librados: complete_aio_write 0x55659dbc95a0 2026-02-25T15:02:16.773 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2026-02-25T15:02:16.773 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d88111880 20 librados: queue_aio_write 0x55659d920390 completion 0x55659da6df30 write_seq 8 2026-02-25T15:02:16.773 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [writefull 0~5 in=5b] tid 34 osd.10 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 34 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d88111880 15 client.4859.objecter _send_op 34 to 25.1 on osd.10 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:34 25.1 25:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55659da6db40 con 0x55659da66160 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.725+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 25 ==== osd_op_reply(34 exist [writefull 0~5] v33'9 uv9 ondisk = 0) ==== 149+0+0 (unknown 627098185 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(34 exist [writefull 0~5] v33'9 uv9 ondisk = 0) 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 34 ondisk uv 9 in 25.1 attempt 0 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 34 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d863ca640 15 client.4859.objecter _finish_op 34 2026-02-25T15:02:16.774 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.864 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 34 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (47 ms) 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d863ca640 20 librados: complete_aio_write 0x55659da6df30 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d88111880 20 librados: queue_aio_write 0x55659d920390 completion 0x55659dbc95a0 write_seq 9 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [writefull 0~5 in=5b] tid 35 osd.10 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 35 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d88111880 15 client.4859.objecter _send_op 35 to 25.1 on osd.10 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:35 25.1 25:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55659da6db40 con 0x55659da66160 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.773+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 26 ==== osd_op_reply(35 exist [writefull 0~5] v33'10 uv10 ondisk = 0) ==== 149+0+0 (unknown 3101756981 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(35 exist [writefull 0~5] v33'10 uv10 ondisk = 0) 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 35 ondisk uv 10 in 25.1 attempt 0 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 35 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d863ca640 15 client.4859.objecter _finish_op 35 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 35 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d863ca640 20 librados: complete_aio_write 0x55659dbc95a0 2026-02-25T15:02:16.865 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (91 ms) 2026-02-25T15:02:16.888 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationAwaitable 2026-02-25T15:02:16.888 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d88111880 20 librados: queue_aio_write 0x55659d920390 completion 0x55659da6df30 write_seq 10 2026-02-25T15:02:16.888 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [writefull 0~5 in=5b] tid 36 osd.10 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 36 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d88111880 15 client.4859.objecter _send_op 36 to 25.1 on osd.10 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:36 25.1 25:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55659da6db40 con 0x55659da66160 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.864+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 27 ==== osd_op_reply(36 exist [writefull 0~5] v33'11 uv11 ondisk = 0) ==== 149+0+0 (unknown 3428955617 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(36 exist [writefull 0~5] v33'11 uv11 ondisk = 0) 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 36 ondisk uv 11 in 25.1 attempt 0 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 36 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d863ca640 15 client.4859.objecter _finish_op 36 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 36 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d863ca640 20 librados: complete_aio_write 0x55659da6df30 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationAwaitable (23 ms) 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncNotifyCallback 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 10 client.4859.objecter linger_register info 0x55659da6c190 linger_id 1 cookie 93894925009296 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 15 client.4859.objecter _session_linger_op_assign 10 1 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 15 client.4859.objecter send_linger 1 register 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6d280 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [notify cookie 93894925009296 in=17b] tid 37 osd.10 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 37 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 15 client.4859.objecter _send_op 37 to 25.1 on osd.10 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:37 25.1 25:8049a5cb:::exist:head [notify cookie 93894925009296 in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55659da6db40 con 0x55659da66160 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.889 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.890 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 10 client.4859.objecter linger_register info 0x55659dbcbc40 linger_id 2 cookie 93894926449728 2026-02-25T15:02:16.890 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.890 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.890 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.890 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.890 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.890 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 15 client.4859.objecter _session_linger_op_assign 10 2 2026-02-25T15:02:16.890 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.890 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 15 client.4859.objecter send_linger 2 register 2026-02-25T15:02:16.890 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659dbcd2c0 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [notify cookie 93894926449728 in=17b] tid 38 osd.10 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 38 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 15 client.4859.objecter _send_op 38 to 25.1 on osd.10 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:38 25.1 25:855dde50:::noexist:head [notify cookie 93894926449728 in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55659dbcd840 con 0x55659da66160 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 7 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.888+0000 7f0d88111880 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 28 ==== watch-notify(notify_complete (2) cookie 93894925009296 notify 141733920768 ret 0) ==== 42+0+8 (unknown 1572923840 0 0) 0x7f0d64001a50 con 0x55659da66160 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 watch-notify(notify_complete (2) cookie 93894925009296 notify 141733920768 ret 0) 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d777fe640 10 librados: operator() completed notify (linger op 0x55659da6c190), ec = system:0 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 29 ==== osd_op_reply(37 exist [notify cookie 93894925009296 out=8b] v0'0 uv11 ondisk = 0) ==== 149+0+8 (unknown 1004810094 0 174112307) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(37 exist [notify cookie 93894925009296 out=8b] v0'0 uv11 ondisk = 0) 2026-02-25T15:02:16.911 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 37 ondisk uv 11 in 25.1 attempt 0 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 8 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 37 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 15 client.4859.objecter _finish_op 37 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 7 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 37 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d77fff640 10 client.4859.objecter _linger_commit 1 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d77fff640 10 client.4859.objecter _linger_commit notify_id=141733920768 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 30 ==== osd_op_reply(38 noexist [notify cookie 93894926449728] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 3075977615 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d77fff640 10 librados: finish linger op 0x55659da6c190 acked (0) 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d77fff640 20 client.4859.objecter _linger_cancel linger_id=1 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(38 noexist [notify cookie 93894926449728] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d77fff640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d77fff640 15 client.4859.objecter _session_linger_op_remove 10 1 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 38 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 10 client.4859.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 38 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 15 client.4859.objecter _finish_op 38 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 38 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d777fe640 10 client.4859.objecter _linger_commit 2 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d777fe640 10 librados: finish linger op 0x55659dbcbc40 acked (-2) 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d777fe640 10 librados: operator() completed notify (linger op 0x55659dbcbc40), ec = osd:2 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncNotifyCallback (22 ms) 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncNotifyDeferred 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.910+0000 7f0d777fe640 20 client.4859.objecter _linger_cancel linger_id=2 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d777fe640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d777fe640 15 client.4859.objecter _session_linger_op_remove 10 2 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 10 client.4859.objecter linger_register info 0x55659da6c190 linger_id 3 cookie 93894925009296 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.912 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 15 client.4859.objecter _session_linger_op_assign 10 3 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 15 client.4859.objecter send_linger 3 register 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6d280 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [notify cookie 93894925009296 in=17b] tid 39 osd.10 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 39 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 15 client.4859.objecter _send_op 39 to 25.1 on osd.10 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:39 25.1 25:8049a5cb:::exist:head [notify cookie 93894925009296 in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55659dbcd840 con 0x55659da66160 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 10 client.4859.objecter linger_register info 0x55659dbccb40 linger_id 4 cookie 93894926453568 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 15 client.4859.objecter _session_linger_op_assign 10 4 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 15 client.4859.objecter send_linger 4 register 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659dbcd260 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [notify cookie 93894926453568 in=17b] tid 40 osd.10 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 40 2026-02-25T15:02:16.913 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 15 client.4859.objecter _send_op 40 to 25.1 on osd.10 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:40 25.1 25:855dde50:::noexist:head [notify cookie 93894926453568 in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55659dbcd840 con 0x55659da66160 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 7 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.911+0000 7f0d88111880 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 31 ==== watch-notify(notify_complete (2) cookie 93894925009296 notify 141733920769 ret 0) ==== 42+0+8 (unknown 3884919630 0 0) 0x7f0d64001a50 con 0x55659da66160 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 watch-notify(notify_complete (2) cookie 93894925009296 notify 141733920769 ret 0) 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 32 ==== osd_op_reply(39 exist [notify cookie 93894925009296 out=8b] v0'0 uv11 ondisk = 0) ==== 149+0+8 (unknown 1004810094 0 1130152724) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(39 exist [notify cookie 93894925009296 out=8b] v0'0 uv11 ondisk = 0) 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 39 ondisk uv 11 in 25.1 attempt 0 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 8 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d77fff640 10 librados: operator() completed notify (linger op 0x55659da6c190), ec = system:0 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 39 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 15 client.4859.objecter _finish_op 39 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 7 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 39 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 33 ==== osd_op_reply(40 noexist [notify cookie 93894926453568] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 2886100963 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(40 noexist [notify cookie 93894926453568] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d777fe640 10 client.4859.objecter _linger_commit 3 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 40 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 10 client.4859.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 40 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 15 client.4859.objecter _finish_op 40 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d777fe640 10 client.4859.objecter _linger_commit notify_id=141733920769 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 40 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d777fe640 10 client.4859.objecter _linger_commit 4 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d77fff640 10 librados: finish linger op 0x55659da6c190 acked (0) 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d77fff640 20 client.4859.objecter _linger_cancel linger_id=3 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d77fff640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d77fff640 15 client.4859.objecter _session_linger_op_remove 10 3 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d777fe640 10 librados: finish linger op 0x55659dbccb40 acked (-2) 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d77fff640 10 librados: operator() completed notify (linger op 0x55659dbccb40), ec = osd:2 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d777fe640 20 client.4859.objecter _linger_cancel linger_id=4 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncNotifyDeferred (6 ms) 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d777fe640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d777fe640 15 client.4859.objecter _session_linger_op_remove 10 4 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncNotifyFuture 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 10 client.4859.objecter linger_register info 0x55659da6c190 linger_id 5 cookie 93894925009296 2026-02-25T15:02:16.918 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.919 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.919 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.919 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.919 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.919 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 15 client.4859.objecter _session_linger_op_assign 10 5 2026-02-25T15:02:16.919 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.919 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 15 client.4859.objecter send_linger 5 register 2026-02-25T15:02:16.919 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6d280 2026-02-25T15:02:16.919 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.919 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.919 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.919 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [notify cookie 93894925009296 in=17b] tid 41 osd.10 2026-02-25T15:02:16.922 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.922 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 41 2026-02-25T15:02:16.922 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 15 client.4859.objecter _send_op 41 to 25.1 on osd.10 2026-02-25T15:02:16.922 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:41 25.1 25:8049a5cb:::exist:head [notify cookie 93894925009296 in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55659dbccb40 con 0x55659da66160 2026-02-25T15:02:16.922 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.922 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 10 client.4859.objecter linger_register info 0x55659dbcd240 linger_id 6 cookie 93894926455360 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 15 client.4859.objecter _session_linger_op_assign 10 6 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 15 client.4859.objecter send_linger 6 register 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659dbcd840 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [notify cookie 93894926455360 in=17b] tid 42 osd.10 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 42 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 15 client.4859.objecter _send_op 42 to 25.1 on osd.10 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:42 25.1 25:855dde50:::noexist:head [notify cookie 93894926455360 in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55659dbccb40 con 0x55659da66160 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 7 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.917+0000 7f0d88111880 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 34 ==== watch-notify(notify_complete (2) cookie 93894925009296 notify 141733920770 ret 0) ==== 42+0+8 (unknown 749935149 0 0) 0x7f0d64001a50 con 0x55659da66160 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 watch-notify(notify_complete (2) cookie 93894925009296 notify 141733920770 ret 0) 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d77fff640 10 librados: operator() completed notify (linger op 0x55659da6c190), ec = system:0 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 35 ==== osd_op_reply(41 exist [notify cookie 93894925009296 out=8b] v0'0 uv11 ondisk = 0) ==== 149+0+8 (unknown 1004810094 0 2551727229) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(41 exist [notify cookie 93894925009296 out=8b] v0'0 uv11 ondisk = 0) 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 41 ondisk uv 11 in 25.1 attempt 0 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 8 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 41 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 15 client.4859.objecter _finish_op 41 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 7 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 41 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 36 ==== osd_op_reply(42 noexist [notify cookie 93894926455360] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 2475933049 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d777fe640 10 client.4859.objecter _linger_commit 5 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(42 noexist [notify cookie 93894926455360] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 42 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d777fe640 10 client.4859.objecter _linger_commit notify_id=141733920770 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 10 client.4859.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 42 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 15 client.4859.objecter _finish_op 42 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.923 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 42 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d777fe640 10 librados: finish linger op 0x55659da6c190 acked (0) 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d777fe640 10 client.4859.objecter _linger_commit 6 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d77fff640 20 client.4859.objecter _linger_cancel linger_id=5 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d77fff640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d77fff640 15 client.4859.objecter _session_linger_op_remove 10 5 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d777fe640 10 librados: finish linger op 0x55659dbcd240 acked (-2) 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d777fe640 10 librados: operator() completed notify (linger op 0x55659dbcd240), ec = osd:2 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d777fe640 20 client.4859.objecter _linger_cancel linger_id=6 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d777fe640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d777fe640 15 client.4859.objecter _session_linger_op_remove 10 6 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncNotifyFuture (4 ms) 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncNotifyYield 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 10 client.4859.objecter linger_register info 0x55659da6c190 linger_id 7 cookie 93894925009296 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 15 client.4859.objecter _session_linger_op_assign 10 7 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 15 client.4859.objecter send_linger 7 register 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6d280 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [notify cookie 93894925009296 in=17b] tid 43 osd.10 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 43 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 15 client.4859.objecter _send_op 43 to 25.1 on osd.10 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:43 25.1 25:8049a5cb:::exist:head [notify cookie 93894925009296 in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55659dbccb40 con 0x55659da66160 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.924 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 10 client.4859.objecter linger_register info 0x55659dbcaad0 linger_id 8 cookie 93894926445264 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 15 client.4859.objecter _session_linger_op_assign 10 8 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 15 client.4859.objecter send_linger 8 register 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659dbcaf00 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [notify cookie 93894926445264 in=17b] tid 44 osd.10 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 44 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 15 client.4859.objecter _send_op 44 to 25.1 on osd.10 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:44 25.1 25:855dde50:::noexist:head [notify cookie 93894926445264 in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55659dbccb40 con 0x55659da66160 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 7 2026-02-25T15:02:16.929 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.922+0000 7f0d88111880 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 37 ==== watch-notify(notify_complete (2) cookie 93894925009296 notify 141733920771 ret 0) ==== 42+0+8 (unknown 2533152931 0 0) 0x7f0d64001a50 con 0x55659da66160 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 watch-notify(notify_complete (2) cookie 93894925009296 notify 141733920771 ret 0) 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 38 ==== osd_op_reply(43 exist [notify cookie 93894925009296 out=8b] v0'0 uv11 ondisk = 0) ==== 149+0+8 (unknown 1004810094 0 3508812122) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(43 exist [notify cookie 93894925009296 out=8b] v0'0 uv11 ondisk = 0) 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 43 ondisk uv 11 in 25.1 attempt 0 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d77fff640 10 librados: operator() completed notify (linger op 0x55659da6c190), ec = system:0 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 8 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 43 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 15 client.4859.objecter _finish_op 43 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 7 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 43 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 39 ==== osd_op_reply(44 noexist [notify cookie 93894926445264] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 2196278408 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d777fe640 10 client.4859.objecter _linger_commit 7 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(44 noexist [notify cookie 93894926445264] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 44 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 10 client.4859.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d777fe640 10 client.4859.objecter _linger_commit notify_id=141733920771 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 44 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 15 client.4859.objecter _finish_op 44 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 44 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d777fe640 10 librados: finish linger op 0x55659da6c190 acked (0) 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d777fe640 10 client.4859.objecter _linger_commit 8 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d77fff640 20 client.4859.objecter _linger_cancel linger_id=7 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d77fff640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d77fff640 15 client.4859.objecter _session_linger_op_remove 10 7 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d777fe640 10 librados: finish linger op 0x55659dbcaad0 acked (-2) 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d777fe640 10 librados: operator() completed notify (linger op 0x55659dbcaad0), ec = osd:2 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d777fe640 20 client.4859.objecter _linger_cancel linger_id=8 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d777fe640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncNotifyYield (6 ms) 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d777fe640 15 client.4859.objecter _session_linger_op_remove 10 8 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncNotifyAwaitable 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 10 client.4859.objecter linger_register info 0x55659da6c190 linger_id 9 cookie 93894925009296 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.d3a59201 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.930 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 15 client.4859.objecter _session_linger_op_assign 10 9 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 15 client.4859.objecter send_linger 9 register 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659dbccb40 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base exist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter _calc_target target exist @25 -> pgid 25.d3a59201 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 10 client.4859.objecter _op_submit oid exist '@25' '@25' [notify cookie 93894925009296 in=17b] tid 45 osd.10 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 45 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 15 client.4859.objecter _send_op 45 to 25.1 on osd.10 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:45 25.1 25:8049a5cb:::exist:head [notify cookie 93894925009296 in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55659da6d280 con 0x55659da66160 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 10 client.4859.objecter linger_register info 0x55659dbcaad0 linger_id 10 cookie 93894926445264 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.931 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 5 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 15 client.4859.objecter _session_linger_op_assign 10 10 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 15 client.4859.objecter send_linger 10 register 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659dbcaf00 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [notify cookie 93894926445264 in=17b] tid 46 osd.10 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 46 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 15 client.4859.objecter _send_op 46 to 25.1 on osd.10 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:46 25.1 25:855dde50:::noexist:head [notify cookie 93894926445264 in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55659dbcb3b0 con 0x55659da66160 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 7 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.928+0000 7f0d88111880 5 client.4859.objecter 2 in flight 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 40 ==== watch-notify(notify_complete (2) cookie 93894925009296 notify 141733920772 ret 0) ==== 42+0+8 (unknown 3207007770 0 0) 0x7f0d64001a50 con 0x55659da66160 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 watch-notify(notify_complete (2) cookie 93894925009296 notify 141733920772 ret 0) 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 41 ==== osd_op_reply(45 exist [notify cookie 93894925009296 out=8b] v0'0 uv11 ondisk = 0) ==== 149+0+8 (unknown 1004810094 0 729627742) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(45 exist [notify cookie 93894925009296 out=8b] v0'0 uv11 ondisk = 0) 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 45 ondisk uv 11 in 25.1 attempt 0 2026-02-25T15:02:16.936 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d77fff640 10 librados: operator() completed notify (linger op 0x55659da6c190), ec = system:0 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 8 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 45 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 15 client.4859.objecter _finish_op 45 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 7 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 45 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 42 ==== osd_op_reply(46 noexist [notify cookie 93894926445264] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 2196278408 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(46 noexist [notify cookie 93894926445264] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d777fe640 10 client.4859.objecter _linger_commit 9 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 46 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 10 client.4859.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d777fe640 10 client.4859.objecter _linger_commit notify_id=141733920772 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 46 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 15 client.4859.objecter _finish_op 46 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 6 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 46 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d777fe640 10 librados: finish linger op 0x55659da6c190 acked (0) 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d777fe640 10 client.4859.objecter _linger_commit 10 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d77fff640 20 client.4859.objecter _linger_cancel linger_id=9 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d77fff640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d77fff640 15 client.4859.objecter _session_linger_op_remove 10 9 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d777fe640 10 librados: finish linger op 0x55659dbcaad0 acked (-2) 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d777fe640 10 librados: operator() completed notify (linger op 0x55659dbcaad0), ec = osd:2 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d777fe640 20 client.4859.objecter _linger_cancel linger_id=10 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d777fe640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d777fe640 15 client.4859.objecter _session_linger_op_remove 10 10 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncNotifyAwaitable (6 ms) 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCancelTerminal 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter chose random osd.10 of [10,13] 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [stat] tid 47 osd.10 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 47 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 15 client.4859.objecter _send_op 47 to 25.1 on osd.10 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:47 25.1 25:855dde50:::noexist:head [stat] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6d280 con 0x55659da66160 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 5 client.4859.objecter _op_cancel: cancelling tid 47 r=-125 2026-02-25T15:02:16.937 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter op_cancel tid 47 in session 10 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 15 client.4859.objecter _finish_op 47 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCancelTerminal (0 ms) 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCancelTotal 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 15 client.4859.objecter _session_op_remove 10 47 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_read 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter chose random osd.13 of [10,13] 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [stat] tid 48 osd.13 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 13 48 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 15 client.4859.objecter _send_op 48 to 25.1 on osd.13 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.93:6812/1850919728 -- osd_op(unknown.0.0:48 25.1 25:855dde50:::noexist:head [stat] snapc 0=[] ondisk+read+balance_reads+known_if_redirected+supports_pool_eio e33) -- 0x55659da6d280 con 0x55659da68610 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 4 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 5 client.4859.objecter _op_cancel: cancelling tid 48 r=-125 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCancelTotal (0 ms) 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCancelTerminal 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter op_cancel tid 48 in session 13 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 15 client.4859.objecter _finish_op 48 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 3 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 15 client.4859.objecter _session_op_remove 13 48 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 librados: queue_aio_write 0x55659d920390 completion 0x55659dbc95a0 write_seq 11 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [stat] tid 49 osd.10 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 49 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 15 client.4859.objecter _send_op 49 to 25.1 on osd.10 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:49 25.1 25:855dde50:::noexist:head [stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55659dbcbb30 con 0x55659da66160 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCancelTerminal (0 ms) 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCancelTotal 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 5 client.4859.objecter _op_cancel: cancelling tid 49 r=-125 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter op_cancel tid 49 in session 10 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 librados: complete_aio_write 0x55659dbc95a0 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 15 client.4859.objecter _finish_op 49 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.938 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 15 client.4859.objecter _session_op_remove 10 49 2026-02-25T15:02:16.939 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 librados: queue_aio_write 0x55659d920390 completion 0x55659dbc95a0 write_seq 12 2026-02-25T15:02:16.939 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter _op_submit op 0x55659da6c190 2026-02-25T15:02:16.939 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter _calc_target epoch 33 base noexist @25 precalc_pgid 0 pgid 0.0 is_write 2026-02-25T15:02:16.939 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter _calc_target target noexist @25 -> pgid 25.a7bbaa1 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter _calc_target raw pgid 25.a7bbaa1 -> actual 25.1 acting [10,13] primary 10 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter _get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 10 client.4859.objecter _op_submit oid noexist '@25' '@25' [stat] tid 50 osd.10 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter get_session s=0x55659da64570 osd=10 3 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 15 client.4859.objecter _session_op_assign 10 50 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 15 client.4859.objecter _send_op 50 to 25.1 on osd.10 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 --> v1:10.20.193.94:6804/4185080112 -- osd_op(unknown.0.0:50 25.1 25:855dde50:::noexist:head [stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55659dbcbb30 con 0x55659da66160 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.935+0000 7f0d88111880 5 client.4859.objecter 1 in flight 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.938+0000 7f0d85bc9640 1 -- 10.20.193.189:0/3178462082 <== osd.13 v1:10.20.193.93:6812/1850919728 10 ==== osd_op_reply(48 noexist [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 2843856563 0 0) 0x7f0d7c002c80 con 0x55659da68610 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.938+0000 7f0d85bc9640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(48 noexist [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.938+0000 7f0d85bc9640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.938+0000 7f0d85bc9640 7 client.4859.objecter handle_osd_op_reply 48 ondisk ... stray 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.938+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 43 ==== osd_op_reply(47 noexist [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 2843856563 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.938+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(47 noexist [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.938+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.938+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 47 ondisk ... stray 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 44 ==== osd_op_reply(49 noexist [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 589447926 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(49 noexist [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 49 ondisk ... stray 2026-02-25T15:02:16.941 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 <== osd.10 v1:10.20.193.94:6804/4185080112 45 ==== osd_op_reply(50 noexist [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (unknown 589447926 0 0) 0x7f0d78001b30 con 0x55659da66160 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d863ca640 10 client.4859.objecter ms_dispatch2 0x55659d93c590 osd_op_reply(50 noexist [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d863ca640 10 client.4859.objecter in handle_osd_op_reply 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d863ca640 7 client.4859.objecter handle_osd_op_reply 50 ondisk uv 0 in 25.1 attempt 0 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d863ca640 10 client.4859.objecter op 0 rval 0 len 0 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d863ca640 15 client.4859.objecter handle_osd_op_reply completed tid 50 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d863ca640 15 client.4859.objecter _finish_op 50 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d863ca640 20 client.4859.objecter put_session s=0x55659da64570 osd=10 4 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d863ca640 15 client.4859.objecter _session_op_remove 10 50 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d863ca640 5 client.4859.objecter 0 in flight 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d863ca640 20 librados: complete_aio_write 0x55659dbc95a0 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCancelTotal (5 ms) 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d88111880 10 librados: watch_flush enter 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d88111880 10 librados: watch_flush exit 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d88111880 10 client.4859.objecter close_session for osd.10 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 >> v1:10.20.193.94:6804/4185080112 conn(0x55659da66160 legacy=0x55659da66ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da64570 osd=10 1 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d88111880 10 client.4859.objecter close_session for osd.13 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 >> v1:10.20.193.93:6812/1850919728 conn(0x55659da68610 legacy=0x55659da68a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d88111880 20 client.4859.objecter put_session s=0x55659da669f0 osd=13 1 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d88111880 20 client.4859.objecter shutdown clearing up homeless session... 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d88111880 10 client.4859.objecter successfully canceled tick 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 >> v1:10.20.193.189:6800/3647375982 conn(0x7f0d64078dc0 legacy=0x7f0d64099270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.940+0000 7f0d88111880 10 monclient: shutdown 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.941+0000 7f0d88111880 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.941+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 >> v1:10.20.193.94:6789/0 conn(0x55659da98b20 legacy=0x55659d9ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.941+0000 7f0d863ca640 1 -- 10.20.193.189:0/3178462082 reap_dead start 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.941+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 shutdown_connections 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.941+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 >> 10.20.193.189:0/3178462082 conn(0x55659d9c7970 msgr2=0x55659da5a410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.941+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 shutdown_connections 2026-02-25T15:02:16.942 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.941+0000 7f0d88111880 1 -- 10.20.193.189:0/3178462082 wait complete. 2026-02-25T15:02:16.944 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.941+0000 7f0d88111880 1 librados: shutdown 2026-02-25T15:02:16.944 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [----------] 29 tests from AsioRados (597 ms total) 2026-02-25T15:02:16.944 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: 2026-02-25T15:02:16.944 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [----------] Global test environment tear-down 2026-02-25T15:02:16.944 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [==========] 29 tests from 1 test suite ran. (6335 ms total) 2026-02-25T15:02:16.944 INFO:tasks.workunit.client.0.trial189.stdout: api_asio: [ PASSED ] 29 tests. 2026-02-25T15:02:16.945 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:02:16.945 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49331 2026-02-25T15:02:16.945 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49331 2026-02-25T15:02:18.458 INFO:tasks.workunit.client.0.trial189.stdout: handler_error: Running main() from gmock_main.cc 2026-02-25T15:02:18.458 INFO:tasks.workunit.client.0.trial189.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2026-02-25T15:02:18.458 INFO:tasks.workunit.client.0.trial189.stdout: handler_error: [----------] Global test environment set-up. 2026-02-25T15:02:18.459 INFO:tasks.workunit.client.0.trial189.stdout: handler_error: [----------] 1 test from neocls_handler_error 2026-02-25T15:02:18.459 INFO:tasks.workunit.client.0.trial189.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2026-02-25T15:02:18.459 INFO:tasks.workunit.client.0.trial189.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (7823 ms) 2026-02-25T15:02:18.459 INFO:tasks.workunit.client.0.trial189.stdout: handler_error: [----------] 1 test from neocls_handler_error (7823 ms total) 2026-02-25T15:02:18.459 INFO:tasks.workunit.client.0.trial189.stdout: handler_error: 2026-02-25T15:02:18.459 INFO:tasks.workunit.client.0.trial189.stdout: handler_error: [----------] Global test environment tear-down 2026-02-25T15:02:18.459 INFO:tasks.workunit.client.0.trial189.stdout: handler_error: [==========] 1 test from 1 test suite ran. (7823 ms total) 2026-02-25T15:02:18.459 INFO:tasks.workunit.client.0.trial189.stdout: handler_error: [ PASSED ] 1 test. 2026-02-25T15:02:18.491 INFO:tasks.workunit.client.0.trial189.stdout: cls: Running main() from gmock_main.cc 2026-02-25T15:02:18.491 INFO:tasks.workunit.client.0.trial189.stdout: cls: [==========] Running 1 test from 1 test suite. 2026-02-25T15:02:18.491 INFO:tasks.workunit.client.0.trial189.stdout: cls: [----------] Global test environment set-up. 2026-02-25T15:02:18.491 INFO:tasks.workunit.client.0.trial189.stdout: cls: [----------] 1 test from NeoRadosCls 2026-02-25T15:02:18.491 INFO:tasks.workunit.client.0.trial189.stdout: cls: [ RUN ] NeoRadosCls.DNE 2026-02-25T15:02:18.491 INFO:tasks.workunit.client.0.trial189.stdout: cls: [ OK ] NeoRadosCls.DNE (7859 ms) 2026-02-25T15:02:18.491 INFO:tasks.workunit.client.0.trial189.stdout: cls: [----------] 1 test from NeoRadosCls (7859 ms total) 2026-02-25T15:02:18.491 INFO:tasks.workunit.client.0.trial189.stdout: cls: 2026-02-25T15:02:18.491 INFO:tasks.workunit.client.0.trial189.stdout: cls: [----------] Global test environment tear-down 2026-02-25T15:02:18.491 INFO:tasks.workunit.client.0.trial189.stdout: cls: [==========] 1 test from 1 test suite ran. (7859 ms total) 2026-02-25T15:02:18.492 INFO:tasks.workunit.client.0.trial189.stdout: cls: [ PASSED ] 1 test. 2026-02-25T15:02:19.537 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd_pp: Running main() from gmock_main.cc 2026-02-25T15:02:19.537 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2026-02-25T15:02:19.537 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd_pp: [----------] Global test environment set-up. 2026-02-25T15:02:19.538 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2026-02-25T15:02:19.538 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2026-02-25T15:02:19.538 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (454 ms) 2026-02-25T15:02:19.538 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2026-02-25T15:02:19.538 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (90 ms) 2026-02-25T15:02:19.538 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2026-02-25T15:02:19.539 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (8386 ms) 2026-02-25T15:02:19.539 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (8931 ms total) 2026-02-25T15:02:19.539 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd_pp: 2026-02-25T15:02:19.539 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd_pp: [----------] Global test environment tear-down 2026-02-25T15:02:19.539 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (8931 ms total) 2026-02-25T15:02:19.540 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2026-02-25T15:02:19.560 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: Running main() from gmock_main.cc 2026-02-25T15:02:19.560 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2026-02-25T15:02:19.560 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [----------] Global test environment set-up. 2026-02-25T15:02:19.560 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2026-02-25T15:02:19.561 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2026-02-25T15:02:19.561 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2026-02-25T15:02:19.561 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2026-02-25T15:02:19.561 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (2634 ms) 2026-02-25T15:02:19.561 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2026-02-25T15:02:19.561 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (255 ms) 2026-02-25T15:02:19.562 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2026-02-25T15:02:19.562 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (121 ms) 2026-02-25T15:02:19.562 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2026-02-25T15:02:19.562 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (604 ms) 2026-02-25T15:02:19.562 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2026-02-25T15:02:19.562 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (108 ms) 2026-02-25T15:02:19.563 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2026-02-25T15:02:19.563 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (87 ms) 2026-02-25T15:02:19.563 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2026-02-25T15:02:19.563 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (238 ms) 2026-02-25T15:02:19.563 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2026-02-25T15:02:19.563 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (105 ms) 2026-02-25T15:02:19.563 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2026-02-25T15:02:19.564 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (36 ms) 2026-02-25T15:02:19.564 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2026-02-25T15:02:19.564 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (57 ms) 2026-02-25T15:02:19.564 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2026-02-25T15:02:19.564 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (54 ms) 2026-02-25T15:02:19.564 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2026-02-25T15:02:19.564 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (52 ms) 2026-02-25T15:02:19.565 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2026-02-25T15:02:19.565 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (201 ms) 2026-02-25T15:02:19.565 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2026-02-25T15:02:19.565 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (117 ms) 2026-02-25T15:02:19.565 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2026-02-25T15:02:19.565 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (142 ms) 2026-02-25T15:02:19.565 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2026-02-25T15:02:19.566 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (50 ms) 2026-02-25T15:02:19.566 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (4870 ms total) 2026-02-25T15:02:19.566 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: 2026-02-25T15:02:19.566 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [----------] Global test environment tear-down 2026-02-25T15:02:19.566 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (8944 ms total) 2026-02-25T15:02:19.566 INFO:tasks.workunit.client.0.trial189.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2026-02-25T15:02:20.232 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: Running main() from gmock_main.cc 2026-02-25T15:02:20.232 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2026-02-25T15:02:20.232 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: [----------] Global test environment set-up. 2026-02-25T15:02:20.232 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2026-02-25T15:02:20.233 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2026-02-25T15:02:20.233 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (528 ms) 2026-02-25T15:02:20.233 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2026-02-25T15:02:20.233 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (31 ms) 2026-02-25T15:02:20.233 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2026-02-25T15:02:20.233 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (8550 ms) 2026-02-25T15:02:20.233 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2026-02-25T15:02:20.233 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:15.982147+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/3453925834' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritetrial189-50100-1"} : dispatch 2026-02-25T15:02:20.233 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:15.985854+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/4245247507' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_trial189-49231-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2026-02-25T15:02:20.233 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:15.990163+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/3113535624' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_trial189-49221-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2026-02-25T15:02:20.234 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:15.997510+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/1483274870' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-trial189-49451-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2026-02-25T15:02:20.234 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:18.146737+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/3453925834' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritetrial189-50100-1"}]': finished 2026-02-25T15:02:20.234 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:18.148877+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/4245247507' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_trial189-49231-2","app": "rados","yes_i_really_mean_it": true}]': finished 2026-02-25T15:02:20.234 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:18.150323+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/3113535624' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_trial189-49221-2","app": "rados","yes_i_really_mean_it": true}]': finished 2026-02-25T15:02:20.234 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:18.151797+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/1483274870' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-trial189-49451-2","app": "rados","yes_i_really_mean_it": true}]': finished 2026-02-25T15:02:20.234 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:18.633957+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/3453925834' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritetrial189-50100-1"} : dispatch 2026-02-25T15:02:20.234 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.420287+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/3453925834' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritetrial189-50100-1"}]': finished 2026-02-25T15:02:20.234 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.557845+0000 mon.b [INF] from='client.? v1:10.20.193.189:0/498595034' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-trial189-49785-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2026-02-25T15:02:20.234 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.565379+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/4245247507' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_trial189-49231-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2026-02-25T15:02:20.234 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.580033+0000 mon.a [INF] from='client.5011 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-trial189-49785-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2026-02-25T15:02:20.234 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.592237+0000 mon.b [INF] from='client.? v1:10.20.193.189:0/4158507043' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOptrial189-50100-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2026-02-25T15:02:20.235 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.604779+0000 mon.a [INF] from='client.5029 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOptrial189-50100-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2026-02-25T15:02:20.235 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.657438+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/2097133905' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_trial189-49541-7"} : dispatch 2026-02-25T15:02:20.235 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.668878+0000 mon.b [INF] from='client.? v1:10.20.193.189:0/3067212529' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_trial189-49583-7"} : dispatch 2026-02-25T15:02:20.235 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.706911+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/2097133905' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_trial189-49541-7"} : dispatch 2026-02-25T15:02:20.235 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.719136+0000 mon.a [INF] from='client.5056 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_trial189-49583-7"} : dispatch 2026-02-25T15:02:20.235 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.734223+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/2097133905' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_trial189-49541-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2026-02-25T15:02:20.235 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.735859+0000 mon.b [INF] from='client.? v1:10.20.193.189:0/3067212529' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_trial189-49583-7"} : dispatch 2026-02-25T15:02:20.235 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.746523+0000 mon.a [INF] from='client.5056 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_trial189-49583-7"} : dispatch 2026-02-25T15:02:20.235 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.764090+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/2734145088' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_trial189-49261-16"} : dispatch 2026-02-25T15:02:20.235 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.767915+0000 mon.b [INF] from='client.? v1:10.20.193.189:0/3067212529' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_trial189-49583-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2026-02-25T15:02:20.235 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.779181+0000 mon.a [INF] from='client.5056 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_trial189-49583-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2026-02-25T15:02:20.235 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.782954+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/2734145088' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_trial189-49261-16"} : dispatch 2026-02-25T15:02:20.235 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.787528+0000 mon.b [INF] from='client.? v1:10.20.193.189:0/3309474439' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2026-02-25T15:02:20.236 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.789609+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/2734145088' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_trial189-49261-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2026-02-25T15:02:23.736 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.799855+0000 client.admin [INF] onexx 2026-02-25T15:02:23.736 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:19.802392+0000 mon.a [INF] from='client.5101 ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2026-02-25T15:02:23.736 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:21.196198+0000 mon.a [INF] from='client.5101 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2026-02-25T15:02:23.736 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:21.607741+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/3941064746' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-trial189-49451-3","app": "rados","yes_i_really_mean_it": true}]': finished 2026-02-25T15:02:23.736 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:21.610255+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/4245247507' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_trial189-49231-3", "field": "max_bytes", "val": "4096"}]': finished 2026-02-25T15:02:23.736 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:21.611382+0000 mon.a [INF] from='client.5009 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_trial189-49363-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2026-02-25T15:02:23.736 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:21.612424+0000 mon.a [INF] from='client.5012 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_trial189-49334-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2026-02-25T15:02:23.736 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:21.845848+0000 mon.c [INF] from='client.? v1:10.20.193.189:0/3466067619' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_trial189-49334-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_trial189-49334-10"} : dispatch 2026-02-25T15:02:23.736 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:21.855613+0000 mon.c [INF] from='client.? v1:10.20.193.189:0/2144538420' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_trial189-49363-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_trial189-49363-10"} : dispatch 2026-02-25T15:02:23.736 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:21.992426+0000 mon.a [INF] from='client.5012 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_trial189-49334-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_trial189-49334-10"} : dispatch 2026-02-25T15:02:23.736 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:22.017236+0000 mon.a [INF] from='client.5009 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_trial189-49363-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_trial189-49363-10"} : dispatch 2026-02-25T15:02:23.736 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:22.657873+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/2097133905' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_trial189-49541-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_trial189-49541-7"}]': finished 2026-02-25T15:02:23.736 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:22.658912+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/2734145088' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_trial189-49261-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_trial189-49261-16"}]': finished 2026-02-25T15:02:23.736 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:22.659769+0000 mon.a [INF] from='client.5056 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_trial189-49583-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_trial189-49583-7"}]': finished 2026-02-25T15:02:23.864 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:22.660835+0000 mon.a [INF] from='client.5029 ' entity='client.admin' cmd='[{"prefix": "osd pool cre cmd: Running main() from gmock_main.cc 2026-02-25T15:02:23.864 INFO:tasks.workunit.client.0.trial189.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2026-02-25T15:02:23.864 INFO:tasks.workunit.client.0.trial189.stdout: cmd: [----------] Global test environment set-up. 2026-02-25T15:02:23.864 INFO:tasks.workunit.client.0.trial189.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2026-02-25T15:02:23.864 INFO:tasks.workunit.client.0.trial189.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2026-02-25T15:02:23.864 INFO:tasks.workunit.client.0.trial189.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (2190 ms) 2026-02-25T15:02:23.864 INFO:tasks.workunit.client.0.trial189.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2026-02-25T15:02:23.864 INFO:tasks.workunit.client.0.trial189.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (3413 ms) 2026-02-25T15:02:23.864 INFO:tasks.workunit.client.0.trial189.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2026-02-25T15:02:23.864 INFO:tasks.workunit.client.0.trial189.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (7628 ms) 2026-02-25T15:02:23.864 INFO:tasks.workunit.client.0.trial189.stdout: cmd: [----------] 3 tests from NeoRadosCmd (13231 ms total) 2026-02-25T15:02:23.864 INFO:tasks.workunit.client.0.trial189.stdout: cmd: 2026-02-25T15:02:23.864 INFO:tasks.workunit.client.0.trial189.stdout: cmd: [----------] Global test environment tear-down 2026-02-25T15:02:23.864 INFO:tasks.workunit.client.0.trial189.stdout: cmd: [==========] 3 tests from 1 test suite ran. (13231 ms total) 2026-02-25T15:02:23.864 INFO:tasks.workunit.client.0.trial189.stdout: cmd: [ PASSED ] 3 tests. 2026-02-25T15:02:25.944 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_1_[49980]: starting. 2026-02-25T15:02:25.945 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_1_[49980]: creating pool ceph_test_rados_open_pools_parallel.trial189-49871 2026-02-25T15:02:25.945 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_1_[49980]: created object 0... 2026-02-25T15:02:25.945 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_1_[49980]: created object 25... 2026-02-25T15:02:25.945 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_1_[49980]: created object 49... 2026-02-25T15:02:25.945 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_1_[49980]: finishing. 2026-02-25T15:02:25.945 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_1_[49980]: shutting down. 2026-02-25T15:02:25.946 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_2_[49986]: starting. 2026-02-25T15:02:25.946 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_2_[49986]: rados_pool_create. 2026-02-25T15:02:25.946 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_2_[49986]: rados_ioctx_create. 2026-02-25T15:02:25.946 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_2_[49986]: shutting down. 2026-02-25T15:02:25.946 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: ******************************* 2026-02-25T15:02:25.946 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_3_[51437]: starting. 2026-02-25T15:02:25.946 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_3_[51437]: creating pool ceph_test_rados_open_pools_parallel.trial189-49871 2026-02-25T15:02:25.946 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_3_[51437]: created object 0... 2026-02-25T15:02:25.946 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_3_[51437]: created object 25... 2026-02-25T15:02:25.946 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_3_[51437]: created object 49... 2026-02-25T15:02:25.947 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_3_[51437]: finishing. 2026-02-25T15:02:25.947 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_3_[51437]: shutting down. 2026-02-25T15:02:25.947 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: ******************************* 2026-02-25T15:02:25.947 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_4_[51438]: starting. 2026-02-25T15:02:25.947 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_4_[51438]: rados_pool_create. 2026-02-25T15:02:25.947 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_4_[51438]: rados_ioctx_create. 2026-02-25T15:02:25.947 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: process_4_[51438]: shutting down. 2026-02-25T15:02:25.947 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: ******************************* 2026-02-25T15:02:25.947 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: ******************************* 2026-02-25T15:02:25.947 INFO:tasks.workunit.client.0.trial189.stdout: open_pools_parallel: ******* SUCCESS ********** 2026-02-25T15:02:26.752 INFO:tasks.workunit.client.0.trial189.stdout:ate", "pool": "ReadOptrial189-50100-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOptrial189-50100-2"}]': finished 2026-02-25T15:02:26.753 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:22.758893+0000 mon.b [INF] from='client.? v1:10.20.193.189:0/1166761150' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-trial189-49785-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2026-02-25T15:02:26.753 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:22.766609+0000 mon.a [INF] from='client.5137 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-trial189-49785-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2026-02-25T15:02:26.753 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:22.769349+0000 mon.b [INF] from='client.? v1:10.20.193.189:0/2141710610' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectstrial189-50144-1"} : dispatch 2026-02-25T15:02:26.753 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:22.783271+0000 mon.a [INF] from='client.? v1:10.20.193.189:0/1651703645' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_trial189-49221-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2026-02-25T15:02:26.753 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:22.789706+0000 mon.b [INF] from='client.? v1:10.20.193.189:0/1042947799' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_trial189-49660-1"} : dispatch 2026-02-25T15:02:26.753 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:22.792362+0000 mon.a [INF] from='client.4894 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectstrial189-50144-1"} : dispatch 2026-02-25T15:02:26.753 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:22.805354+0000 mon.a [INF] from='client.4849 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_trial189-49660-1"} : dispatch 2026-02-25T15:02:26.753 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:23.241729+0000 mon.b [INF] from='client.? v1:10.20.193.189:0/3309474439' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2026-02-25T15:02:26.753 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:23.245414+0000 client.admin [INF] threexx 2026-02-25T15:02:26.753 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: got: 2026-02-25T15:02:23.247417+0000 mon.a [INF] from='client.5101 ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2026-02-25T15:02:26.754 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (7039 ms) 2026-02-25T15:02:26.754 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (16149 ms total) 2026-02-25T15:02:26.754 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: 2026-02-25T15:02:26.754 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: [----------] Global test environment tear-down 2026-02-25T15:02:26.754 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (16149 ms total) 2026-02-25T15:02:26.754 INFO:tasks.workunit.client.0.trial189.stdout: api_cmd: [ PASSED ] 4 tests. 2026-02-25T15:02:27.004 INFO:tasks.workunit.client.0.trial189.stdout: ec_io: Running main() from gmock_main.cc 2026-02-25T15:02:27.005 INFO:tasks.workunit.client.0.trial189.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2026-02-25T15:02:27.005 INFO:tasks.workunit.client.0.trial189.stdout: ec_io: [----------] Global test environment set-up. 2026-02-25T15:02:27.005 INFO:tasks.workunit.client.0.trial189.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2026-02-25T15:02:27.005 INFO:tasks.workunit.client.0.trial189.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2026-02-25T15:02:27.005 INFO:tasks.workunit.client.0.trial189.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (8795 ms) 2026-02-25T15:02:27.005 INFO:tasks.workunit.client.0.trial189.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2026-02-25T15:02:27.005 INFO:tasks.workunit.client.0.trial189.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (7567 ms) 2026-02-25T15:02:27.005 INFO:tasks.workunit.client.0.trial189.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (16362 ms total) 2026-02-25T15:02:27.005 INFO:tasks.workunit.client.0.trial189.stdout: ec_io: 2026-02-25T15:02:27.006 INFO:tasks.workunit.client.0.trial189.stdout: ec_io: [----------] Global test environment tear-down 2026-02-25T15:02:27.006 INFO:tasks.workunit.client.0.trial189.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (16362 ms total) 2026-02-25T15:02:27.006 INFO:tasks.workunit.client.0.trial189.stdout: ec_io: [ PASSED ] 2 tests. 2026-02-25T15:02:27.024 INFO:tasks.workunit.client.0.trial189.stdout: pool: Running main() from gmock_main.cc 2026-02-25T15:02:27.025 INFO:tasks.workunit.client.0.trial189.stdout: pool: [==========] Running 6 tests from 1 test suite. 2026-02-25T15:02:27.025 INFO:tasks.workunit.client.0.trial189.stdout: pool: [----------] Global test environment set-up. 2026-02-25T15:02:27.025 INFO:tasks.workunit.client.0.trial189.stdout: pool: [----------] 6 tests from NeoRadosPools 2026-02-25T15:02:27.025 INFO:tasks.workunit.client.0.trial189.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2026-02-25T15:02:27.025 INFO:tasks.workunit.client.0.trial189.stdout: pool: [ OK ] NeoRadosPools.PoolList (2159 ms) 2026-02-25T15:02:27.025 INFO:tasks.workunit.client.0.trial189.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2026-02-25T15:02:27.025 INFO:tasks.workunit.client.0.trial189.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (3441 ms) 2026-02-25T15:02:27.025 INFO:tasks.workunit.client.0.trial189.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2026-02-25T15:02:27.025 INFO:tasks.workunit.client.0.trial189.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (4373 ms) 2026-02-25T15:02:27.025 INFO:tasks.workunit.client.0.trial189.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2026-02-25T15:02:27.025 INFO:tasks.workunit.client.0.trial189.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (2137 ms) 2026-02-25T15:02:27.026 INFO:tasks.workunit.client.0.trial189.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2026-02-25T15:02:27.026 INFO:tasks.workunit.client.0.trial189.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (2145 ms) 2026-02-25T15:02:27.026 INFO:tasks.workunit.client.0.trial189.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2026-02-25T15:02:27.026 INFO:tasks.workunit.client.0.trial189.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (2112 ms) 2026-02-25T15:02:27.026 INFO:tasks.workunit.client.0.trial189.stdout: pool: [----------] 6 tests from NeoRadosPools (16370 ms total) 2026-02-25T15:02:27.026 INFO:tasks.workunit.client.0.trial189.stdout: pool: 2026-02-25T15:02:27.026 INFO:tasks.workunit.client.0.trial189.stdout: pool: [----------] Global test environment tear-down 2026-02-25T15:02:27.026 INFO:tasks.workunit.client.0.trial189.stdout: pool: [==========] 6 tests from 1 test suite ran. (16370 ms total) 2026-02-25T15:02:27.026 INFO:tasks.workunit.client.0.trial189.stdout: pool: [ PASSED ] 6 tests. 2026-02-25T15:02:27.027 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_1_[50042]: starting. 2026-02-25T15:02:27.027 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_1_[50042]: creating pool ceph_test_rados_delete_pools_parallel.trial189-49927 2026-02-25T15:02:27.027 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_1_[50042]: created object 0... 2026-02-25T15:02:27.027 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_1_[50042]: created object 25... 2026-02-25T15:02:27.027 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_1_[50042]: created object 49... 2026-02-25T15:02:27.027 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_1_[50042]: finishing. 2026-02-25T15:02:27.027 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_1_[50042]: shutting down. 2026-02-25T15:02:27.027 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_2_[50048]: starting. 2026-02-25T15:02:27.027 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_2_[50048]: deleting pool ceph_test_rados_delete_pools_parallel.trial189-49927 2026-02-25T15:02:27.027 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_2_[50048]: shutting down. 2026-02-25T15:02:27.028 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: ******************************* 2026-02-25T15:02:27.028 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_3_[51324]: starting. 2026-02-25T15:02:27.028 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_3_[51324]: creating pool ceph_test_rados_delete_pools_parallel.trial189-49927 2026-02-25T15:02:27.028 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_3_[51324]: created object 0... 2026-02-25T15:02:27.028 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_3_[51324]: created object 25... 2026-02-25T15:02:27.028 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_3_[51324]: created object 49... 2026-02-25T15:02:27.028 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_3_[51324]: finishing. 2026-02-25T15:02:27.028 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_3_[51324]: shutting down. 2026-02-25T15:02:27.028 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: ******************************* 2026-02-25T15:02:27.028 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_5_[51326]: starting. 2026-02-25T15:02:27.028 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_5_[51326]: listing objects. 2026-02-25T15:02:27.028 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_5_[51326]: listed object 0... 2026-02-25T15:02:27.028 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_5_[51326]: listed object 25... 2026-02-25T15:02:27.029 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_5_[51326]: saw 50 objects 2026-02-25T15:02:27.029 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_5_[51326]: shutting down. 2026-02-25T15:02:27.029 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: ******************************* 2026-02-25T15:02:27.029 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_4_[51325]: starting. 2026-02-25T15:02:27.029 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_4_[51325]: deleting pool ceph_test_rados_delete_pools_parallel.trial189-49927 2026-02-25T15:02:27.029 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: process_4_[51325]: shutting down. 2026-02-25T15:02:27.029 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: ******************************* 2026-02-25T15:02:27.029 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: ******************************* 2026-02-25T15:02:27.029 INFO:tasks.workunit.client.0.trial189.stdout: delete_pools_parallel: ******* SUCCESS ********** 2026-02-25T15:02:28.130 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: Running main() from gmock_main.cc 2026-02-25T15:02:28.130 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2026-02-25T15:02:28.130 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [----------] Global test environment set-up. 2026-02-25T15:02:28.130 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2026-02-25T15:02:28.130 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: seed 49583 2026-02-25T15:02:28.130 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2026-02-25T15:02:28.130 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (2644 ms) 2026-02-25T15:02:28.130 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2026-02-25T15:02:28.130 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (172 ms) 2026-02-25T15:02:28.131 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2026-02-25T15:02:28.131 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (576 ms) 2026-02-25T15:02:28.131 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2026-02-25T15:02:28.131 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (931 ms) 2026-02-25T15:02:28.131 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2026-02-25T15:02:28.131 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (124 ms) 2026-02-25T15:02:28.131 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (4450 ms total) 2026-02-25T15:02:28.131 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: 2026-02-25T15:02:28.131 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2026-02-25T15:02:28.131 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2026-02-25T15:02:28.131 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (2099 ms) 2026-02-25T15:02:28.131 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2026-02-25T15:02:28.131 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (59 ms) 2026-02-25T15:02:28.132 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2026-02-25T15:02:28.132 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (656 ms) 2026-02-25T15:02:28.132 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2026-02-25T15:02:28.132 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (193 ms) 2026-02-25T15:02:28.132 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (3008 ms total) 2026-02-25T15:02:28.132 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: 2026-02-25T15:02:28.132 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [----------] Global test environment tear-down 2026-02-25T15:02:28.132 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (17537 ms total) 2026-02-25T15:02:28.132 INFO:tasks.workunit.client.0.trial189.stdout: api_stat_pp: [ PASSED ] 9 tests. 2026-02-25T15:02:28.133 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: Running main() from gmock_main.cc 2026-02-25T15:02:28.133 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2026-02-25T15:02:28.133 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [----------] Global test environment set-up. 2026-02-25T15:02:28.133 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [----------] 5 tests from LibRadosStat 2026-02-25T15:02:28.133 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2026-02-25T15:02:28.133 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ OK ] LibRadosStat.Stat (3372 ms) 2026-02-25T15:02:28.133 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2026-02-25T15:02:28.133 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (248 ms) 2026-02-25T15:02:28.133 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2026-02-25T15:02:28.134 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ OK ] LibRadosStat.StatNS (124 ms) 2026-02-25T15:02:28.134 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2026-02-25T15:02:28.134 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (1046 ms) 2026-02-25T15:02:28.134 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2026-02-25T15:02:28.134 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (37 ms) 2026-02-25T15:02:28.134 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [----------] 5 tests from LibRadosStat (4830 ms total) 2026-02-25T15:02:28.134 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: 2026-02-25T15:02:28.134 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2026-02-25T15:02:28.134 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2026-02-25T15:02:28.134 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (2053 ms) 2026-02-25T15:02:28.134 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2026-02-25T15:02:28.134 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (787 ms) 2026-02-25T15:02:28.134 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2026-02-25T15:02:28.135 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (1 ms) 2026-02-25T15:02:28.135 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2026-02-25T15:02:28.135 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (128 ms) 2026-02-25T15:02:28.135 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (2970 ms total) 2026-02-25T15:02:28.135 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: 2026-02-25T15:02:28.135 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [----------] Global test environment tear-down 2026-02-25T15:02:28.135 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (17541 ms total) 2026-02-25T15:02:28.135 INFO:tasks.workunit.client.0.trial189.stdout: api_stat: [ PASSED ] 9 tests. 2026-02-25T15:02:29.167 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2026-02-25T15:02:29.167 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [----------] Global test environment set-up. 2026-02-25T15:02:29.167 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [----------] 7 tests from LibRadosList 2026-02-25T15:02:29.167 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2026-02-25T15:02:29.167 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ OK ] LibRadosList.ListObjects (4003 ms) 2026-02-25T15:02:29.167 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2026-02-25T15:02:29.167 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (360 ms) 2026-02-25T15:02:29.167 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2026-02-25T15:02:29.167 INFO:tasks.workunit.client.0.trial189.stdout: api_list: myset foo1,foo2,foo3 2026-02-25T15:02:29.167 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo1 2026-02-25T15:02:29.167 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo2 2026-02-25T15:02:29.167 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo3 2026-02-25T15:02:29.167 INFO:tasks.workunit.client.0.trial189.stdout: api_list: myset foo1,foo4,foo5 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo4 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo5 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo1 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: myset foo6,foo7 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo7 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo6 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: ns1:foo4 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: ns1:foo5 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: ns2:foo7 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: ns2:foo6 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: ns1:foo1 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: :foo1 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: :foo2 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: :foo3 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (1960 ms) 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 1 0 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 10 0 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 13 0 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 7 0 2026-02-25T15:02:29.168 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 14 0 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 0 0 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 15 0 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 11 0 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 5 0 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 8 0 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 6 0 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 3 0 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 4 0 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 12 0 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 9 0 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 2 0 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (4555 ms) 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: x cursor=MIN 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=1 cursor=8:02547ec2:::1:head 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=10 cursor=8:52ea6a34:::10:head 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=13 cursor=8:566253c9:::13:head 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=7 cursor=8:5c6b0b28:::7:head 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=14 cursor=8:62a1935d:::14:head 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=0 cursor=8:6cac518f:::0:head 2026-02-25T15:02:29.169 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=15 cursor=8:863748b0:::15:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=11 cursor=8:89d3ae78:::11:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=5 cursor=8:b29083e3:::5:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=8 cursor=8:bd63b0f1:::8:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=6 cursor=8:c4fdafeb:::6:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=3 cursor=8:cfc208b3:::3:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=4 cursor=8:d83876eb:::4:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=12 cursor=8:de5d7c5f:::12:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=9 cursor=8:e960b815:::9:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > oid=2 cursor=8:f905c69b:::2:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: FIRST> seek to MIN oid=1 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=1 cursor=8:02547ec2:::1:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:02547ec2:::1:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:02547ec2:::1:head -> 1 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=10 cursor=8:52ea6a34:::10:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:52ea6a34:::10:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:52ea6a34:::10:head -> 10 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=13 cursor=8:566253c9:::13:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:566253c9:::13:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:566253c9:::13:head -> 13 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=7 cursor=8:5c6b0b28:::7:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:5c6b0b28:::7:head 2026-02-25T15:02:29.170 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:5c6b0b28:::7:head -> 7 2026-02-25T15:02:30.654 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=14 cursor=8:62a1935d:::14:he api_io: Running main() from gmock_main.cc 2026-02-25T15:02:30.654 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2026-02-25T15:02:30.654 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [----------] Global test environment set-up. 2026-02-25T15:02:30.654 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [----------] 14 tests from LibRadosIo 2026-02-25T15:02:30.654 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2026-02-25T15:02:30.654 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (2469 ms) 2026-02-25T15:02:30.655 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2026-02-25T15:02:30.655 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2026-02-25T15:02:30.655 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2026-02-25T15:02:30.655 INFO:tasks.workunit.client.0.trial189.stdout: api_io: no timeout :/ 2026-02-25T15:02:30.655 INFO:tasks.workunit.client.0.trial189.stdout: api_io: no timeout :/ 2026-02-25T15:02:30.655 INFO:tasks.workunit.client.0.trial189.stdout: api_io: no timeout :/ 2026-02-25T15:02:30.656 INFO:tasks.workunit.client.0.trial189.stdout: api_io: no timeout :/ 2026-02-25T15:02:30.656 INFO:tasks.workunit.client.0.trial189.stdout: api_io: no timeout :/ 2026-02-25T15:02:30.656 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (1926 ms) 2026-02-25T15:02:30.656 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2026-02-25T15:02:30.656 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (106 ms) 2026-02-25T15:02:30.656 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2026-02-25T15:02:30.656 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIo.Checksum (36 ms) 2026-02-25T15:02:30.656 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2026-02-25T15:02:30.656 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (50 ms) 2026-02-25T15:02:30.656 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2026-02-25T15:02:30.657 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (66 ms) 2026-02-25T15:02:30.657 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2026-02-25T15:02:30.657 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (74 ms) 2026-02-25T15:02:30.657 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2026-02-25T15:02:30.657 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (46 ms) 2026-02-25T15:02:30.657 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2026-02-25T15:02:30.657 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIo.TruncTest (43 ms) 2026-02-25T15:02:30.657 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2026-02-25T15:02:30.657 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (37 ms) 2026-02-25T15:02:30.657 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2026-02-25T15:02:30.657 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (60 ms) 2026-02-25T15:02:30.657 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2026-02-25T15:02:30.658 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIo.RmXattr (172 ms) 2026-02-25T15:02:30.658 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2026-02-25T15:02:30.658 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIo.XattrIter (109 ms) 2026-02-25T15:02:30.658 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [----------] 14 tests from LibRadosIo (5200 ms total) 2026-02-25T15:02:30.658 INFO:tasks.workunit.client.0.trial189.stdout: api_io: 2026-02-25T15:02:30.658 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2026-02-25T15:02:30.658 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2026-02-25T15:02:30.658 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1953 ms) 2026-02-25T15:02:30.658 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2026-02-25T15:02:30.658 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (623 ms) 2026-02-25T15:02:30.658 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2026-02-25T15:02:30.658 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (427 ms) 2026-02-25T15:02:30.658 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2026-02-25T15:02:30.659 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (132 ms) 2026-02-25T15:02:30.659 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2026-02-25T15:02:30.659 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (796 ms) 2026-02-25T15:02:30.659 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2026-02-25T15:02:30.659 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (744 ms) 2026-02-25T15:02:30.659 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2026-02-25T15:02:30.659 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (121 ms) 2026-02-25T15:02:30.659 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2026-02-25T15:02:30.659 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (83 ms) 2026-02-25T15:02:30.659 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2026-02-25T15:02:30.659 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (272 ms) 2026-02-25T15:02:30.659 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2026-02-25T15:02:30.659 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (125 ms) 2026-02-25T15:02:30.660 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [----------] 10 tests from LibRadosIoEC (5280 ms total) 2026-02-25T15:02:30.660 INFO:tasks.workunit.client.0.trial189.stdout: api_io: 2026-02-25T15:02:30.660 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [----------] Global test environment tear-down 2026-02-25T15:02:30.660 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [==========] 24 tests from 2 test suites ran. (20095 ms total) 2026-02-25T15:02:30.660 INFO:tasks.workunit.client.0.trial189.stdout: api_io: [ PASSED ] 24 tests. 2026-02-25T15:02:30.733 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:01:44.240 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:30.738 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:01:44.245 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:30.768 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:01:44.237 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:30.770 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:01:44.240 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:30.774 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:01:44.276 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:30.778 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:01:44.281 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:30.819 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:01:44.326 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:30.850 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:01:44.353 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:30.859 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:01:44.328 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:02:31.474 INFO:tasks.workunit.client.0.trial189.stdout:ad 2026-02-25T15:02:31.474 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:62a1935d:::14:head 2026-02-25T15:02:31.474 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:62a1935d:::14:head -> 14 2026-02-25T15:02:31.474 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=0 cursor=8:6cac518f:::0:head 2026-02-25T15:02:31.474 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:6cac518f:::0:head 2026-02-25T15:02:31.474 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:6cac518f:::0:head -> 0 2026-02-25T15:02:31.475 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=15 cursor=8:863748b0:::15:head 2026-02-25T15:02:31.475 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:863748b0:::15:head 2026-02-25T15:02:31.475 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:863748b0:::15:head -> 15 2026-02-25T15:02:31.475 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=11 cursor=8:89d3ae78:::11:head 2026-02-25T15:02:31.475 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:89d3ae78:::11:head 2026-02-25T15:02:31.475 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:89d3ae78:::11:head -> 11 2026-02-25T15:02:31.475 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=5 cursor=8:b29083e3:::5:head 2026-02-25T15:02:31.475 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:b29083e3:::5:head 2026-02-25T15:02:31.475 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:b29083e3:::5:head -> 5 2026-02-25T15:02:31.475 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=8 cursor=8:bd63b0f1:::8:head 2026-02-25T15:02:31.475 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:bd63b0f1:::8:head 2026-02-25T15:02:31.475 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:bd63b0f1:::8:head -> 8 2026-02-25T15:02:31.475 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=6 cursor=8:c4fdafeb:::6:head 2026-02-25T15:02:31.476 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:c4fdafeb:::6:head 2026-02-25T15:02:31.476 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:c4fdafeb:::6:head -> 6 2026-02-25T15:02:31.476 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=3 cursor=8:cfc208b3:::3:head 2026-02-25T15:02:31.476 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:cfc208b3:::3:head 2026-02-25T15:02:31.476 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:cfc208b3:::3:head -> 3 2026-02-25T15:02:31.476 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=4 cursor=8:d83876eb:::4:head 2026-02-25T15:02:31.476 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:d83876eb:::4:head 2026-02-25T15:02:31.476 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:d83876eb:::4:head -> 4 2026-02-25T15:02:31.476 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=12 cursor=8:de5d7c5f:::12:head 2026-02-25T15:02:31.476 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:de5d7c5f:::12:head 2026-02-25T15:02:31.476 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:de5d7c5f:::12:head -> 12 2026-02-25T15:02:31.476 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=9 cursor=8:e960b815:::9:head 2026-02-25T15:02:31.476 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:e960b815:::9:head 2026-02-25T15:02:31.477 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:e960b815:::9:head -> 9 2026-02-25T15:02:31.477 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : oid=2 cursor=8:f905c69b:::2:head 2026-02-25T15:02:31.477 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:f905c69b:::2:head 2026-02-25T15:02:31.477 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:f905c69b:::2:head -> 2 2026-02-25T15:02:31.477 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:f905c69b:::2:head 2026-02-25T15:02:31.477 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : cursor()=8:f905c69b:::2:head expected=8:f905c69b:::2:head 2026-02-25T15:02:31.477 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:f905c69b:::2:head -> 2 2026-02-25T15:02:31.477 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=2 expected=2 2026-02-25T15:02:31.477 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:d83876eb:::4:head 2026-02-25T15:02:31.477 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : cursor()=8:d83876eb:::4:head expected=8:d83876eb:::4:head 2026-02-25T15:02:31.477 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:d83876eb:::4:head -> 4 2026-02-25T15:02:31.477 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=4 expected=4 2026-02-25T15:02:31.478 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:cfc208b3:::3:head 2026-02-25T15:02:31.478 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : cursor()=8:cfc208b3:::3:head expected=8:cfc208b3:::3:head 2026-02-25T15:02:31.478 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:cfc208b3:::3:head -> 3 2026-02-25T15:02:31.478 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=3 expected=3 2026-02-25T15:02:31.478 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:c4fdafeb:::6:head 2026-02-25T15:02:31.478 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : cursor()=8:c4fdafeb:::6:head expected=8:c4fdafeb:::6:head 2026-02-25T15:02:31.478 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:c4fdafeb:::6:head -> 6 2026-02-25T15:02:31.478 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=6 expected=6 2026-02-25T15:02:31.478 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:bd63b0f1:::8:head 2026-02-25T15:02:31.478 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : cursor()=8:bd63b0f1:::8:head expected=8:bd63b0f1:::8:head 2026-02-25T15:02:31.478 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:bd63b0f1:::8:head -> 8 2026-02-25T15:02:31.478 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=8 expected=8 2026-02-25T15:02:31.479 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:b29083e3:::5:head 2026-02-25T15:02:31.479 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : cursor()=8:b29083e3:::5:head expected=8:b29083e3:::5:head 2026-02-25T15:02:31.479 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:b29083e3:::5:head -> 5 2026-02-25T15:02:31.479 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=5 expected=5 2026-02-25T15:02:31.479 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:89d3ae78:::11:head 2026-02-25T15:02:31.479 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : cursor()=8:89d3ae78:::11:head expected=8:89d3ae78:::11:head 2026-02-25T15:02:31.479 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:89d3ae78:::11:head -> 11 2026-02-25T15:02:31.479 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=11 expected=11 2026-02-25T15:02:31.479 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:02547ec2:::1:head 2026-02-25T15:02:31.479 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : cursor()=8:02547ec2:::1:head expected=8:02547ec2:::1:head 2026-02-25T15:02:31.479 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:02547ec2:::1:head -> 1 2026-02-25T15:02:31.479 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=1 expected=1 2026-02-25T15:02:31.479 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:e960b815:::9:head 2026-02-25T15:02:32.810 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : api_lock_pp: Running main() from gmock_main.cc 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [----------] Global test environment set-up. 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: seed 49363 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (2658 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (226 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1543 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1097 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (576 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (46 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (97 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (317 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (6565 ms total) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1709 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (115 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1162 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1090 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (119 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (879 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (120 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (93 ms) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (5290 ms total) 2026-02-25T15:02:32.811 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: 2026-02-25T15:02:32.812 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [----------] Global test environment tear-down 2026-02-25T15:02:32.812 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (22237 ms total) 2026-02-25T15:02:32.812 INFO:tasks.workunit.client.0.trial189.stdout: api_lock_pp: [ PASSED ] 16 tests. 2026-02-25T15:02:32.860 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: Running main() from gmock_main.cc 2026-02-25T15:02:32.860 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2026-02-25T15:02:32.860 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [----------] Global test environment set-up. 2026-02-25T15:02:32.860 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [----------] 8 tests from LibRadosLock 2026-02-25T15:02:32.860 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2026-02-25T15:02:32.860 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (2605 ms) 2026-02-25T15:02:32.860 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLock.LockShared (647 ms) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1107 ms) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1389 ms) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (310 ms) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLock.Unlock (55 ms) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (159 ms) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (247 ms) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [----------] 8 tests from LibRadosLock (6523 ms total) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1898 ms) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (121 ms) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1600 ms) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1822 ms) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (454 ms) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (118 ms) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (105 ms) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (58 ms) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (6180 ms total) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [----------] Global test environment tear-down 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (22290 ms total) 2026-02-25T15:02:32.861 INFO:tasks.workunit.client.0.trial189.stdout: api_lock: [ PASSED ] 16 tests. 2026-02-25T15:02:33.607 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: Running main() from gmock_main.cc 2026-02-25T15:02:33.608 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [==========] Running 40 tests from 2 test suites. 2026-02-25T15:02:33.608 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [----------] Global test environment set-up. 2026-02-25T15:02:33.608 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [----------] 22 tests from LibRadosIoPP 2026-02-25T15:02:33.608 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: seed 49274 2026-02-25T15:02:33.608 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2026-02-25T15:02:33.608 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2026-02-25T15:02:33.608 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2026-02-25T15:02:33.608 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (2568 ms) 2026-02-25T15:02:33.608 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2026-02-25T15:02:33.608 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (162 ms) 2026-02-25T15:02:33.608 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2026-02-25T15:02:33.609 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (489 ms) 2026-02-25T15:02:33.609 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadExtentArrayOpPP 2026-02-25T15:02:33.609 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadExtentArrayOpPP (9600 ms) 2026-02-25T15:02:33.609 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2026-02-25T15:02:33.609 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (88 ms) 2026-02-25T15:02:33.609 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2026-02-25T15:02:33.609 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (26 ms) 2026-02-25T15:02:33.609 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2026-02-25T15:02:33.609 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (27 ms) 2026-02-25T15:02:33.609 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2026-02-25T15:02:33.609 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (22 ms) 2026-02-25T15:02:33.609 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2026-02-25T15:02:33.609 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (76 ms) 2026-02-25T15:02:33.610 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2026-02-25T15:02:33.610 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (44 ms) 2026-02-25T15:02:33.610 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2026-02-25T15:02:33.610 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (554 ms) 2026-02-25T15:02:33.610 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2026-02-25T15:02:33.610 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (190 ms) 2026-02-25T15:02:33.610 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2026-02-25T15:02:33.610 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (121 ms) 2026-02-25T15:02:33.610 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2026-02-25T15:02:33.610 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (53 ms) 2026-02-25T15:02:33.610 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2026-02-25T15:02:33.610 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (42 ms) 2026-02-25T15:02:33.611 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2026-02-25T15:02:33.611 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (953 ms) 2026-02-25T15:02:33.611 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2026-02-25T15:02:33.611 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (84 ms) 2026-02-25T15:02:33.611 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CrcZeroWrite 2026-02-25T15:02:33.611 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.CrcZeroWrite (47 ms) 2026-02-25T15:02:33.611 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2026-02-25T15:02:33.611 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (34 ms) 2026-02-25T15:02:33.611 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2026-02-25T15:02:33.611 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (28 ms) 2026-02-25T15:02:33.611 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2026-02-25T15:02:33.611 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (32 ms) 2026-02-25T15:02:33.611 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [----------] 22 tests from LibRadosIoPP (15249 ms total) 2026-02-25T15:02:33.612 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: 2026-02-25T15:02:33.612 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [----------] 18 tests from LibRadosIoECPP 2026-02-25T15:02:33.612 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2026-02-25T15:02:33.612 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1639 ms) 2026-02-25T15:02:33.612 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2026-02-25T15:02:33.612 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (281 ms) 2026-02-25T15:02:33.612 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2026-02-25T15:02:34.309 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadO list_parallel: process_1_[49945]: starting. 2026-02-25T15:02:34.309 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_1_[49945]: creating pool ceph_test_rados_list_parallel.trial189-49836 2026-02-25T15:02:34.310 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_1_[49945]: created object 0... 2026-02-25T15:02:34.310 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_1_[49945]: created object 25... 2026-02-25T15:02:34.310 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_1_[49945]: created object 49... 2026-02-25T15:02:34.310 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_1_[49945]: finishing. 2026-02-25T15:02:34.310 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_1_[49945]: shutting down. 2026-02-25T15:02:34.310 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_2_[49947]: starting. 2026-02-25T15:02:34.310 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_2_[49947]: listing objects. 2026-02-25T15:02:34.310 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_2_[49947]: listed object 0... 2026-02-25T15:02:34.310 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_2_[49947]: listed object 25... 2026-02-25T15:02:34.310 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_2_[49947]: saw 50 objects 2026-02-25T15:02:34.310 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_2_[49947]: shutting down. 2026-02-25T15:02:34.311 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:34.311 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_3_[51195]: starting. 2026-02-25T15:02:34.311 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_3_[51195]: creating pool ceph_test_rados_list_parallel.trial189-49836 2026-02-25T15:02:34.311 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_3_[51195]: created object 0... 2026-02-25T15:02:34.311 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_3_[51195]: created object 25... 2026-02-25T15:02:34.311 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_3_[51195]: created object 49... 2026-02-25T15:02:34.311 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_3_[51195]: finishing. 2026-02-25T15:02:34.311 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_3_[51195]: shutting down. 2026-02-25T15:02:34.311 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:34.311 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_4_[51196]: starting. 2026-02-25T15:02:34.311 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_4_[51196]: listing objects. 2026-02-25T15:02:34.311 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_4_[51196]: listed object 0... 2026-02-25T15:02:34.312 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_4_[51196]: listed object 25... 2026-02-25T15:02:34.312 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_4_[51196]: saw 45 objects 2026-02-25T15:02:34.312 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_4_[51196]: shutting down. 2026-02-25T15:02:34.312 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:34.312 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_5_[51197]: starting. 2026-02-25T15:02:34.312 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_5_[51197]: removed 25 objects... 2026-02-25T15:02:34.312 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_5_[51197]: removed half of the objects 2026-02-25T15:02:34.312 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_5_[51197]: removed 50 objects... 2026-02-25T15:02:34.312 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_5_[51197]: removed 50 objects 2026-02-25T15:02:34.312 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_5_[51197]: shutting down. 2026-02-25T15:02:34.312 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:34.312 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:34.312 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_6_[51949]: starting. 2026-02-25T15:02:34.313 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_6_[51949]: creating pool ceph_test_rados_list_parallel.trial189-49836 2026-02-25T15:02:34.313 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_6_[51949]: created object 0... 2026-02-25T15:02:34.313 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_6_[51949]: created object 25... 2026-02-25T15:02:34.313 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_6_[51949]: created object 49... 2026-02-25T15:02:34.313 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_6_[51949]: finishing. 2026-02-25T15:02:34.313 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_6_[51949]: shutting down. 2026-02-25T15:02:34.313 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:34.313 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:34.313 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_7_[51950]: starting. 2026-02-25T15:02:34.313 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_7_[51950]: listing objects. 2026-02-25T15:02:34.313 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_7_[51950]: listed object 0... 2026-02-25T15:02:34.313 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_7_[51950]: listed object 25... 2026-02-25T15:02:34.313 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_7_[51950]: listed object 50... 2026-02-25T15:02:34.314 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_7_[51950]: saw 54 objects 2026-02-25T15:02:34.314 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_7_[51950]: shutting down. 2026-02-25T15:02:34.314 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:34.314 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:34.314 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_8_[51951]: starting. 2026-02-25T15:02:34.314 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_8_[51951]: added 25 objects... 2026-02-25T15:02:34.314 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_8_[51951]: added half of the objects 2026-02-25T15:02:34.314 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_8_[51951]: added 50 objects... 2026-02-25T15:02:34.314 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_8_[51951]: added 50 objects 2026-02-25T15:02:34.314 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_8_[51951]: shutting down. 2026-02-25T15:02:34.314 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:34.314 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:36.457 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ************************* api_watch_notify_pp: Running main() from gmock_main.cc 2026-02-25T15:02:36.457 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2026-02-25T15:02:36.457 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2026-02-25T15:02:36.457 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2026-02-25T15:02:36.457 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2026-02-25T15:02:36.458 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: notify 2026-02-25T15:02:36.458 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (5372 ms) 2026-02-25T15:02:36.458 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2026-02-25T15:02:36.458 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (2784 ms) 2026-02-25T15:02:36.458 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (8157 ms total) 2026-02-25T15:02:36.458 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: 2026-02-25T15:02:36.458 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2026-02-25T15:02:36.458 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2026-02-25T15:02:36.458 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: notify 2026-02-25T15:02:36.458 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (720 ms) 2026-02-25T15:02:36.458 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2026-02-25T15:02:36.458 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: notify 2026-02-25T15:02:36.459 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3514 ms) 2026-02-25T15:02:36.459 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2026-02-25T15:02:36.459 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (147 ms) 2026-02-25T15:02:36.459 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2026-02-25T15:02:36.459 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (57 ms) 2026-02-25T15:02:36.459 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2026-02-25T15:02:36.459 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: handle_notify cookie 94316782310592 notify_id 197568495617 notifier_gid 5081 2026-02-25T15:02:36.459 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (84 ms) 2026-02-25T15:02:36.459 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2026-02-25T15:02:36.459 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: handle_notify cookie 94316782313856 notify_id 201863462913 notifier_gid 5081 2026-02-25T15:02:36.459 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (717 ms) 2026-02-25T15:02:36.459 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2026-02-25T15:02:36.460 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: handle_notify cookie 94316782313856 notify_id 201863462914 notifier_gid 5081 2026-02-25T15:02:36.460 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (86 ms) 2026-02-25T15:02:36.460 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2026-02-25T15:02:36.460 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: handle_notify cookie 94316782640048 notify_id 201863462913 notifier_gid 5081 2026-02-25T15:02:36.460 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (173 ms) 2026-02-25T15:02:36.460 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2026-02-25T15:02:36.460 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: handle_notify cookie 94316782335824 notify_id 201863462914 notifier_gid 5081 2026-02-25T15:02:36.460 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (88 ms) 2026-02-25T15:02:36.460 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2026-02-25T15:02:36.460 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: handle_notify cookie 94316782335824 notify_id 206158430214 notifier_gid 5081 2026-02-25T15:02:36.460 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (770 ms) 2026-02-25T15:02:36.460 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2026-02-25T15:02:36.460 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: trying... 2026-02-25T15:02:36.461 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: handle_notify cookie 94316782335824 notify_id 206158430211 notifier_gid 5081 2026-02-25T15:02:36.461 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: timed out 2026-02-25T15:02:36.461 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: flushing 2026-02-25T15:02:39.801 INFO:tasks.workunit.client.0.trial189.stdout: ****** 2026-02-25T15:02:39.801 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_9_[52209]: starting. 2026-02-25T15:02:39.801 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_9_[52209]: creating pool ceph_test_rados_list_parallel.trial189-49836 2026-02-25T15:02:39.801 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_9_[52209]: created object 0... 2026-02-25T15:02:39.802 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_9_[52209]: created object 25... 2026-02-25T15:02:39.802 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_9_[52209]: created object 49... 2026-02-25T15:02:39.802 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_9_[52209]: finishing. 2026-02-25T15:02:39.802 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_9_[52209]: shutting down. 2026-02-25T15:02:39.802 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.802 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.802 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.802 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_10_[52210]: starting. 2026-02-25T15:02:39.802 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_10_[52210]: listing objects. 2026-02-25T15:02:39.802 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_10_[52210]: listed object 0... 2026-02-25T15:02:39.802 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_10_[52210]: listed object 25... 2026-02-25T15:02:39.803 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_10_[52210]: listed object 50... 2026-02-25T15:02:39.803 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_10_[52210]: listed object 75... 2026-02-25T15:02:39.803 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_10_[52210]: saw 99 objects 2026-02-25T15:02:39.803 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_10_[52210]: shutting down. 2026-02-25T15:02:39.803 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.803 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.803 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.803 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_13_[52213]: starting. 2026-02-25T15:02:39.803 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_13_[52213]: removed 25 objects... 2026-02-25T15:02:39.803 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_13_[52213]: removed half of the objects 2026-02-25T15:02:39.803 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_13_[52213]: removed 50 objects... 2026-02-25T15:02:39.804 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_13_[52213]: removed 50 objects 2026-02-25T15:02:39.804 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_13_[52213]: shutting down. 2026-02-25T15:02:39.804 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.804 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.804 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.804 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_12_[52212]: starting. 2026-02-25T15:02:39.804 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_12_[52212]: added 25 objects... 2026-02-25T15:02:39.804 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_12_[52212]: added half of the objects 2026-02-25T15:02:39.804 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_12_[52212]: added 50 objects... 2026-02-25T15:02:39.804 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_12_[52212]: added 50 objects 2026-02-25T15:02:39.804 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_12_[52212]: shutting down. 2026-02-25T15:02:39.805 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.805 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.805 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.805 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_11_[52211]: starting. 2026-02-25T15:02:39.805 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_11_[52211]: added 25 objects... 2026-02-25T15:02:39.805 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_11_[52211]: added half of the objects 2026-02-25T15:02:39.805 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_11_[52211]: added 50 objects... 2026-02-25T15:02:39.805 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_11_[52211]: added 50 objects 2026-02-25T15:02:39.805 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_11_[52211]: shutting down. 2026-02-25T15:02:39.805 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.805 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.806 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.806 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.806 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_14_[56231]: starting. 2026-02-25T15:02:39.806 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_14_[56231]: creating pool ceph_test_rados_list_parallel.trial189-49836 2026-02-25T15:02:39.806 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_14_[56231]: created object 0... 2026-02-25T15:02:39.806 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_14_[56231]: created object 25... 2026-02-25T15:02:39.806 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_14_[56231]: created object 49... 2026-02-25T15:02:39.806 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_14_[56231]: finishing. 2026-02-25T15:02:39.806 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_14_[56231]: shutting down. 2026-02-25T15:02:39.806 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.806 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.806 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.806 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:39.807 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_15_[56232]: starting. 2026-02-25T15:02:39.807 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_15_[56232]: listing objects. 2026-02-25T15:02:39.807 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_15_[56232]: listed object 0... 2026-02-25T15:02:39.807 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_15_[56232]: listed object 25... 2026-02-25T15:02:40.763 INFO:tasks.workunit.client.0.trial189.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2026-02-25T15:02:40.763 INFO:tasks.workunit.client.0.trial189.stdout: api_service: [----------] Global test environment set-up. 2026-02-25T15:02:40.763 INFO:tasks.workunit.client.0.trial189.stdout: api_service: [----------] 4 tests from LibRadosService 2026-02-25T15:02:40.764 INFO:tasks.workunit.client.0.trial189.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2026-02-25T15:02:40.764 INFO:tasks.workunit.client.0.trial189.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5331 ms) 2026-02-25T15:02:40.764 INFO:tasks.workunit.client.0.trial189.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2026-02-25T15:02:40.764 INFO:tasks.workunit.client.0.trial189.stdout: api_service: [ OK ] LibRadosService.RegisterLate (140 ms) 2026-02-25T15:02:40.764 INFO:tasks.workunit.client.0.trial189.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2026-02-25T15:02:40.764 INFO:tasks.workunit.client.0.trial189.stdout: api_service: cluster: 2026-02-25T15:02:40.764 INFO:tasks.workunit.client.0.trial189.stdout: api_service: id: 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:02:40.764 INFO:tasks.workunit.client.0.trial189.stdout: api_service: health: HEALTH_OK 2026-02-25T15:02:40.764 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 2026-02-25T15:02:40.764 INFO:tasks.workunit.client.0.trial189.stdout: api_service: services: 2026-02-25T15:02:40.764 INFO:tasks.workunit.client.0.trial189.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 82s) [leader: a] 2026-02-25T15:02:40.764 INFO:tasks.workunit.client.0.trial189.stdout: api_service: mgr: x(active, since 79s), standbys: y 2026-02-25T15:02:40.765 INFO:tasks.workunit.client.0.trial189.stdout: api_service: osd: 16 osds: 16 up (since 49s), 16 in (since 75s) 2026-02-25T15:02:40.765 INFO:tasks.workunit.client.0.trial189.stdout: api_service: laundry: 2 daemons active (1 hosts) 2026-02-25T15:02:40.765 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 2026-02-25T15:02:40.765 INFO:tasks.workunit.client.0.trial189.stdout: api_service: data: 2026-02-25T15:02:40.765 INFO:tasks.workunit.client.0.trial189.stdout: api_service: pools: 35 pools, 993 pgs 2026-02-25T15:02:40.765 INFO:tasks.workunit.client.0.trial189.stdout: api_service: objects: 14 objects, 449 KiB 2026-02-25T15:02:40.765 INFO:tasks.workunit.client.0.trial189.stdout: api_service: usage: 428 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T15:02:40.765 INFO:tasks.workunit.client.0.trial189.stdout: api_service: pgs: 36.254% pgs unknown 2026-02-25T15:02:40.765 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 24.068% pgs not active 2026-02-25T15:02:40.765 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 394 active+clean 2026-02-25T15:02:40.765 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 360 unknown 2026-02-25T15:02:40.765 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 200 creating+activating 2026-02-25T15:02:40.766 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 39 creating+peering 2026-02-25T15:02:40.766 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 2026-02-25T15:02:40.766 INFO:tasks.workunit.client.0.trial189.stdout: api_service: io: 2026-02-25T15:02:40.766 INFO:tasks.workunit.client.0.trial189.stdout: api_service: client: 0 B/s rd, 1023 B/s wr, 0 op/s rd, 1 op/s wr 2026-02-25T15:02:40.766 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 2026-02-25T15:02:40.766 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 2026-02-25T15:02:40.766 INFO:tasks.workunit.client.0.trial189.stdout: api_service: cluster: 2026-02-25T15:02:40.766 INFO:tasks.workunit.client.0.trial189.stdout: api_service: id: 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:02:40.766 INFO:tasks.workunit.client.0.trial189.stdout: api_service: health: HEALTH_OK 2026-02-25T15:02:40.766 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 2026-02-25T15:02:40.766 INFO:tasks.workunit.client.0.trial189.stdout: api_service: services: 2026-02-25T15:02:40.767 INFO:tasks.workunit.client.0.trial189.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 84s) [leader: a] 2026-02-25T15:02:40.767 INFO:tasks.workunit.client.0.trial189.stdout: api_service: mgr: x(active, since 81s), standbys: y 2026-02-25T15:02:40.767 INFO:tasks.workunit.client.0.trial189.stdout: api_service: osd: 16 osds: 16 up (since 51s), 16 in (since 77s) 2026-02-25T15:02:40.767 INFO:tasks.workunit.client.0.trial189.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2026-02-25T15:02:40.767 INFO:tasks.workunit.client.0.trial189.stdout: api_service: laundry: 1 daemon active (1 hosts) 2026-02-25T15:02:40.767 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 2026-02-25T15:02:40.767 INFO:tasks.workunit.client.0.trial189.stdout: api_service: data: 2026-02-25T15:02:40.767 INFO:tasks.workunit.client.0.trial189.stdout: api_service: pools: 29 pools, 825 pgs 2026-02-25T15:02:40.767 INFO:tasks.workunit.client.0.trial189.stdout: api_service: objects: 54 objects, 453 KiB 2026-02-25T15:02:40.767 INFO:tasks.workunit.client.0.trial189.stdout: api_service: usage: 429 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T15:02:40.767 INFO:tasks.workunit.client.0.trial189.stdout: api_service: pgs: 13.455% pgs unknown 2026-02-25T15:02:40.768 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 16.970% pgs not active 2026-02-25T15:02:40.768 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 574 active+clean 2026-02-25T15:02:40.768 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 111 unknown 2026-02-25T15:02:40.768 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 107 creating+activating 2026-02-25T15:02:40.768 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 33 creating+peering 2026-02-25T15:02:40.768 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 2026-02-25T15:02:40.768 INFO:tasks.workunit.client.0.trial189.stdout: api_service: io: 2026-02-25T15:02:40.768 INFO:tasks.workunit.client.0.trial189.stdout: api_service: client: 1.2 KiB/s rd, 8.0 KiB/s wr, 2 op/s rd, 10 op/s wr 2026-02-25T15:02:40.768 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 2026-02-25T15:02:40.768 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 2026-02-25T15:02:40.768 INFO:tasks.workunit.client.0.trial189.stdout: api_service: [ OK ] LibRadosService.StatusFormat (4622 ms) 2026-02-25T15:02:40.768 INFO:tasks.workunit.client.0.trial189.stdout: api_service: [ RUN ] LibRadosService.Status 2026-02-25T15:02:40.769 INFO:tasks.workunit.client.0.trial189.stdout: api_service: [ OK ] LibRadosService.Status (20054 ms) 2026-02-25T15:02:40.769 INFO:tasks.workunit.client.0.trial189.stdout: api_service: [----------] 4 tests from LibRadosService (30149 ms total) 2026-02-25T15:02:40.769 INFO:tasks.workunit.client.0.trial189.stdout: api_service: 2026-02-25T15:02:40.769 INFO:tasks.workunit.client.0.trial189.stdout: api_service: [----------] Global test environment tear-down 2026-02-25T15:02:40.769 INFO:tasks.workunit.client.0.trial189.stdout: api_service: [==========] 4 tests from 1 test suite ran. (30149 ms total) 2026-02-25T15:02:40.769 INFO:tasks.workunit.client.0.trial189.stdout: api_service: [ PASSED ] 4 tests. 2026-02-25T15:02:43.769 INFO:tasks.workunit.client.0.trial189.stdout: list: Running main() from gmock_main.cc 2026-02-25T15:02:43.769 INFO:tasks.workunit.client.0.trial189.stdout: list: [==========] Running 3 tests from 1 test suite. 2026-02-25T15:02:43.769 INFO:tasks.workunit.client.0.trial189.stdout: list: [----------] Global test environment set-up. 2026-02-25T15:02:43.769 INFO:tasks.workunit.client.0.trial189.stdout: list: [----------] 3 tests from NeoradosList 2026-02-25T15:02:43.770 INFO:tasks.workunit.client.0.trial189.stdout: list: [ RUN ] NeoradosList.ListObjects 2026-02-25T15:02:43.770 INFO:tasks.workunit.client.0.trial189.stdout: list: [ OK ] NeoradosList.ListObjects (8915 ms) 2026-02-25T15:02:43.770 INFO:tasks.workunit.client.0.trial189.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2026-02-25T15:02:43.770 INFO:tasks.workunit.client.0.trial189.stdout: list: [ OK ] NeoradosList.ListObjectsNS (6382 ms) 2026-02-25T15:02:43.770 INFO:tasks.workunit.client.0.trial189.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2026-02-25T15:02:43.770 INFO:tasks.workunit.client.0.trial189.stdout: list: [ OK ] NeoradosList.ListObjectsMany (17826 ms) 2026-02-25T15:02:43.770 INFO:tasks.workunit.client.0.trial189.stdout: list: [----------] 3 tests from NeoradosList (33125 ms total) 2026-02-25T15:02:43.770 INFO:tasks.workunit.client.0.trial189.stdout: list: 2026-02-25T15:02:43.770 INFO:tasks.workunit.client.0.trial189.stdout: list: [----------] Global test environment tear-down 2026-02-25T15:02:43.770 INFO:tasks.workunit.client.0.trial189.stdout: list: [==========] 3 tests from 1 test suite ran. (33125 ms total) 2026-02-25T15:02:43.770 INFO:tasks.workunit.client.0.trial189.stdout: list: [ PASSED ] 3 tests. 2026-02-25T15:02:45.655 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_15_[56232]: listed object 50... 2026-02-25T15:02:45.655 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_15_[56232]: listed object 75... 2026-02-25T15:02:45.655 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_15_[56232]: listed object 100... 2026-02-25T15:02:45.656 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_15_[56232]: listed object 125... 2026-02-25T15:02:45.656 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_15_[56232]: saw 150 objects 2026-02-25T15:02:45.656 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_15_[56232]: shutting down. 2026-02-25T15:02:45.656 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:45.656 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:45.656 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:45.656 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:45.656 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_16_[56233]: starting. 2026-02-25T15:02:45.656 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_16_[56233]: added 25 objects... 2026-02-25T15:02:45.656 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_16_[56233]: added half of the objects 2026-02-25T15:02:45.656 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_16_[56233]: added 50 objects... 2026-02-25T15:02:45.657 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_16_[56233]: added 50 objects 2026-02-25T15:02:45.657 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: process_16_[56233]: shutting down. 2026-02-25T15:02:45.657 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:45.657 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:45.657 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:45.657 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:45.657 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******************************* 2026-02-25T15:02:45.657 INFO:tasks.workunit.client.0.trial189.stdout: list_parallel: ******* SUCCESS ********** 2026-02-25T15:02:46.155 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: Running main() from gmock_main.cc 2026-02-25T15:02:46.156 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2026-02-25T15:02:46.156 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [----------] Global test environment set-up. 2026-02-25T15:02:46.156 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2026-02-25T15:02:46.156 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2026-02-25T15:02:46.156 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: watch_notify_test_cb 2026-02-25T15:02:46.156 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (3654 ms) 2026-02-25T15:02:46.156 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2026-02-25T15:02:46.156 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: watch_notify2_test_errcb cookie waiting up to 94061779623904 err -107300 2026-02-25T15:02:46.156 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: for disconnect notification ... 2026-02-25T15:02:46.156 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (128 ms) 2026-02-25T15:02:46.156 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2026-02-25T15:02:46.157 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2026-02-25T15:02:46.157 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94061779631680 err -107 2026-02-25T15:02:46.157 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1146 ms) 2026-02-25T15:02:46.157 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2026-02-25T15:02:46.157 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: watch_notify2_test_cb from 4841 notify_id 141733920768 cookie 94061779634000 2026-02-25T15:02:46.157 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (179 ms) 2026-02-25T15:02:46.157 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2026-02-25T15:02:46.157 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: watch_notify2_test_cb from 4841 notify_id 141733920768 cookie 94061779678816 2026-02-25T15:02:46.157 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (111 ms) 2026-02-25T15:02:46.157 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2026-02-25T15:02:46.157 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: watch_notify2_test_cb from 4841 notify_id 141733920773 cookie 94061779678816 2026-02-25T15:02:46.157 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (64 ms) 2026-02-25T15:02:46.157 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2026-02-25T15:02:46.158 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: watch_notify2_test_cb from 4841 notify_id 141733920769 cookie 94061779678816 2026-02-25T15:02:46.158 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: watch_notify2_test_cb from 4841 notify_id 141733920769 cookie 94061781446640 2026-02-25T15:02:46.158 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (383 ms) 2026-02-25T15:02:46.158 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2026-02-25T15:02:46.158 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: watch_notify2_test_cb from 4841 notify_id 146028888064 cookie 94061779678816 2026-02-25T15:02:46.158 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: watch_notify2_test_cb from 4841 notify_id 150323855361 cookie 94061779678816 2026-02-25T15:02:46.158 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3976 ms) 2026-02-25T15:02:46.158 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2026-02-25T15:02:46.158 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2026-02-25T15:02:46.158 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94061779678816 err -107 2026-02-25T15:02:46.158 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: watch_notify2_test_cb from 4841 notify_id 180388626433 cookie 94061779678816 2026-02-25T15:02:46.158 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5292 ms) 2026-02-25T15:02:46.158 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2026-02-25T15:02:46.159 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2026-02-25T15:02:46.159 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94061781446640 err -107 2026-02-25T15:02:46.159 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1141 ms) 2026-02-25T15:02:46.159 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (16079 ms total) 2026-02-25T15:02:46.159 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: 2026-02-25T15:02:46.159 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2026-02-25T15:02:46.159 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2026-02-25T15:02:46.159 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: watch_notify_test_cb 2026-02-25T15:02:46.159 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1736 ms) 2026-02-25T15:02:46.159 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1736 ms total) 2026-02-25T15:02:46.159 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: 2026-02-25T15:02:46.159 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [----------] Global test environment tear-down 2026-02-25T15:02:46.160 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (35563 ms total) 2026-02-25T15:02:46.160 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify: [ PASSED ] 11 tests. 2026-02-25T15:02:46.933 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2026-02-25T15:02:46.934 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: [----------] Global test environment set-up. 2026-02-25T15:02:46.934 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2026-02-25T15:02:46.934 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2026-02-25T15:02:46.934 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5377 ms) 2026-02-25T15:02:46.934 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2026-02-25T15:02:46.934 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (109 ms) 2026-02-25T15:02:46.934 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2026-02-25T15:02:46.934 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20454 ms) 2026-02-25T15:02:46.934 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2026-02-25T15:02:46.934 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: attempt 0 of 20 2026-02-25T15:02:46.934 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (10376 ms) 2026-02-25T15:02:46.935 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (36317 ms total) 2026-02-25T15:02:46.935 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: 2026-02-25T15:02:46.935 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: [----------] Global test environment tear-down 2026-02-25T15:02:46.935 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (36318 ms total) 2026-02-25T15:02:46.935 INFO:tasks.workunit.client.0.trial189.stdout: api_service_pp: [ PASSED ] 4 tests. 2026-02-25T15:02:55.984 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: Running main() from gmock_main.cc 2026-02-25T15:02:55.984 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [==========] Running 6 tests from 2 test suites. 2026-02-25T15:02:55.984 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [----------] Global test environment set-up. 2026-02-25T15:02:55.984 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2026-02-25T15:02:55.985 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2026-02-25T15:02:55.985 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: handle_notify cookie 93978221935424 notify_id 137438953472 notifier_gid 4897 2026-02-25T15:02:55.985 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (7898 ms) 2026-02-25T15:02:55.985 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2026-02-25T15:02:55.985 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: Trying... 2026-02-25T15:02:55.985 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: handle_notify cookie 93978234762496 notify_id 154618822656 notifier_gid 5023 2026-02-25T15:02:55.985 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: Waiting for 3.000000000s 2026-02-25T15:02:55.985 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: Timed out. 2026-02-25T15:02:55.985 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: Flushing... 2026-02-25T15:02:55.985 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: Flushed... 2026-02-25T15:02:55.985 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (7384 ms) 2026-02-25T15:02:55.986 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (15283 ms total) 2026-02-25T15:02:55.986 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: 2026-02-25T15:02:55.986 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [----------] 4 tests from NeoRadosWatchNotifyPoll 2026-02-25T15:02:55.986 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [ RUN ] NeoRadosWatchNotifyPoll.WatchNotify 2026-02-25T15:02:55.986 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [ OK ] NeoRadosWatchNotifyPoll.WatchNotify (3233 ms) 2026-02-25T15:02:55.986 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [ RUN ] NeoRadosWatchNotifyPoll.WatchNotifyTimeout 2026-02-25T15:02:55.986 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [ OK ] NeoRadosWatchNotifyPoll.WatchNotifyTimeout (3562 ms) 2026-02-25T15:02:55.986 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [ RUN ] NeoRadosWatchNotifyPoll.WrongWatchType 2026-02-25T15:02:55.986 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [ OK ] NeoRadosWatchNotifyPoll.WrongWatchType (11442 ms) 2026-02-25T15:02:55.986 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [ RUN ] NeoRadosWatchNotifyPoll.WatchNotifyCancel 2026-02-25T15:02:55.986 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [ OK ] NeoRadosWatchNotifyPoll.WatchNotifyCancel (11802 ms) 2026-02-25T15:02:55.986 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [----------] 4 tests from NeoRadosWatchNotifyPoll (30040 ms total) 2026-02-25T15:02:55.986 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: 2026-02-25T15:02:55.987 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [----------] Global test environment tear-down 2026-02-25T15:02:55.987 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [==========] 6 tests from 2 test suites ran. (45324 ms total) 2026-02-25T15:02:55.987 INFO:tasks.workunit.client.0.trial189.stdout: watch_notify: [ PASSED ] 6 tests. 2026-02-25T15:02:59.401 INFO:tasks.workunit.client.0.trial189.stdout:api_watch_notify_pp: flushed 2026-02-25T15:02:59.401 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3067 ms) 2026-02-25T15:02:59.401 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2026-02-25T15:02:59.401 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: trying... 2026-02-25T15:02:59.401 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: handle_notify cookie 94316782335824 notify_id 210453397507 notifier_gid 5081 2026-02-25T15:02:59.401 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: timed out 2026-02-25T15:02:59.401 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: flushing 2026-02-25T15:02:59.402 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: flushed 2026-02-25T15:02:59.402 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3047 ms) 2026-02-25T15:02:59.402 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2026-02-25T15:02:59.402 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: List watches 2026-02-25T15:02:59.402 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: notify2 2026-02-25T15:02:59.402 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: handle_notify cookie 94316782347856 notify_id 219043332100 notifier_gid 5081 2026-02-25T15:02:59.402 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: notify2 done 2026-02-25T15:02:59.402 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: watch_check 2026-02-25T15:02:59.402 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: unwatch2 2026-02-25T15:02:59.402 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: flushing 2026-02-25T15:02:59.402 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: done 2026-02-25T15:02:59.402 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3196 ms) 2026-02-25T15:02:59.403 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2026-02-25T15:02:59.403 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: List watches 2026-02-25T15:02:59.403 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: notify2 2026-02-25T15:02:59.403 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: handle_notify cookie 94316782335824 notify_id 223338299396 notifier_gid 5081 2026-02-25T15:02:59.403 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: notify2 done 2026-02-25T15:02:59.403 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: watch_check 2026-02-25T15:02:59.403 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: unwatch2 2026-02-25T15:02:59.403 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: flushing 2026-02-25T15:02:59.403 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: done 2026-02-25T15:02:59.403 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3085 ms) 2026-02-25T15:02:59.403 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (18758 ms total) 2026-02-25T15:02:59.403 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: 2026-02-25T15:02:59.404 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2026-02-25T15:02:59.404 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (48800 ms total) 2026-02-25T15:02:59.404 INFO:tasks.workunit.client.0.trial189.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2026-02-25T15:03:01.780 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:02:15.287 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:01.783 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:02:15.290 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:01.865 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:02:15.372 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:02.098 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:02:15.567 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:02.101 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:02:15.570 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:02.126 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:02:15.628 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:02.129 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:02:15.632 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:02.261 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:02:15.730 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:02.276 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:02:15.778 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:06.743 INFO:tasks.workunit.client.0.trial189.stdout:pPP (47 ms) 2026-02-25T15:03:06.743 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2026-02-25T15:03:06.743 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (40 ms) 2026-02-25T15:03:06.743 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2026-02-25T15:03:06.743 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (39 ms) 2026-02-25T15:03:06.744 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2026-02-25T15:03:06.744 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (86 ms) 2026-02-25T15:03:06.744 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2026-02-25T15:03:06.744 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (176 ms) 2026-02-25T15:03:06.744 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2026-02-25T15:03:06.744 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (36 ms) 2026-02-25T15:03:06.744 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2026-02-25T15:03:06.744 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (99 ms) 2026-02-25T15:03:06.744 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2026-02-25T15:03:06.744 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (51 ms) 2026-02-25T15:03:06.744 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2026-02-25T15:03:06.744 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (71 ms) 2026-02-25T15:03:06.744 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2026-02-25T15:03:06.745 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (65 ms) 2026-02-25T15:03:06.745 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2026-02-25T15:03:06.745 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (255 ms) 2026-02-25T15:03:06.745 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CrcZeroWrite 2026-02-25T15:03:06.745 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CrcZeroWrite (21808 ms) 2026-02-25T15:03:06.745 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2026-02-25T15:03:06.745 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (3784 ms) 2026-02-25T15:03:06.745 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2026-02-25T15:03:06.745 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (280 ms) 2026-02-25T15:03:06.745 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2026-02-25T15:03:06.745 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (130 ms) 2026-02-25T15:03:06.745 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2026-02-25T15:03:06.746 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (65 ms) 2026-02-25T15:03:06.746 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [----------] 18 tests from LibRadosIoECPP (28961 ms total) 2026-02-25T15:03:06.746 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: 2026-02-25T15:03:06.746 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [----------] Global test environment tear-down 2026-02-25T15:03:06.746 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [==========] 40 tests from 2 test suites ran. (56181 ms total) 2026-02-25T15:03:06.746 INFO:tasks.workunit.client.0.trial189.stdout: api_io_pp: [ PASSED ] 40 tests. 2026-02-25T15:03:08.633 INFO:tasks.workunit.client.0.trial189.stdout: ec_list: Running main() from gmock_main.cc 2026-02-25T15:03:08.633 INFO:tasks.workunit.client.0.trial189.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2026-02-25T15:03:08.634 INFO:tasks.workunit.client.0.trial189.stdout: ec_list: [----------] Global test environment set-up. 2026-02-25T15:03:08.634 INFO:tasks.workunit.client.0.trial189.stdout: ec_list: [----------] 3 tests from NeoradosECList 2026-02-25T15:03:08.634 INFO:tasks.workunit.client.0.trial189.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2026-02-25T15:03:08.634 INFO:tasks.workunit.client.0.trial189.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (14256 ms) 2026-02-25T15:03:08.634 INFO:tasks.workunit.client.0.trial189.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2026-02-25T15:03:08.634 INFO:tasks.workunit.client.0.trial189.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (7740 ms) 2026-02-25T15:03:08.634 INFO:tasks.workunit.client.0.trial189.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2026-02-25T15:03:08.634 INFO:tasks.workunit.client.0.trial189.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (35989 ms) 2026-02-25T15:03:08.634 INFO:tasks.workunit.client.0.trial189.stdout: ec_list: [----------] 3 tests from NeoradosECList (57987 ms total) 2026-02-25T15:03:08.634 INFO:tasks.workunit.client.0.trial189.stdout: ec_list: 2026-02-25T15:03:08.634 INFO:tasks.workunit.client.0.trial189.stdout: ec_list: [----------] Global test environment tear-down 2026-02-25T15:03:08.635 INFO:tasks.workunit.client.0.trial189.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (57987 ms total) 2026-02-25T15:03:08.635 INFO:tasks.workunit.client.0.trial189.stdout: ec_list: [ PASSED ] 3 tests. 2026-02-25T15:03:20.514 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: Running main() from gmock_main.cc 2026-02-25T15:03:20.514 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2026-02-25T15:03:20.514 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [----------] Global test environment set-up. 2026-02-25T15:03:20.514 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2026-02-25T15:03:20.514 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2026-02-25T15:03:20.514 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2026-02-25T15:03:20.514 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2026-02-25T15:03:20.515 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: 2026-02-25T15:03:20.515 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2026-02-25T15:03:20.515 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2026-02-25T15:03:20.515 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (7926 ms) 2026-02-25T15:03:20.515 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2026-02-25T15:03:20.515 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (3178 ms) 2026-02-25T15:03:20.515 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2026-02-25T15:03:20.515 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (4220 ms) 2026-02-25T15:03:20.515 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2026-02-25T15:03:20.515 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (4344 ms) 2026-02-25T15:03:20.515 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2026-02-25T15:03:20.515 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (7303 ms) 2026-02-25T15:03:20.516 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2026-02-25T15:03:20.516 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (12631 ms) 2026-02-25T15:03:20.516 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2026-02-25T15:03:20.516 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (30295 ms) 2026-02-25T15:03:20.516 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (69900 ms total) 2026-02-25T15:03:20.516 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: 2026-02-25T15:03:20.516 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [----------] Global test environment tear-down 2026-02-25T15:03:20.516 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (69900 ms total) 2026-02-25T15:03:20.516 INFO:tasks.workunit.client.0.trial189.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2026-02-25T15:03:26.508 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:02:40.015 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:26.516 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:02:40.023 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:26.559 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:02:40.066 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:26.654 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:02:40.123 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:26.688 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:02:40.157 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:26.688 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:02:40.191 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:26.716 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:02:40.185 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:26.788 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:02:40.291 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:26.818 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:02:40.321 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:27.748 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: Running main() from gmock_main.cc 2026-02-25T15:03:27.748 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [==========] Running 8 tests from 1 test suite. 2026-02-25T15:03:27.748 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [----------] Global test environment set-up. 2026-02-25T15:03:27.749 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [----------] 8 tests from NeoRadosWriteOps 2026-02-25T15:03:27.749 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2026-02-25T15:03:27.749 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (7896 ms) 2026-02-25T15:03:27.749 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2026-02-25T15:03:27.749 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (4204 ms) 2026-02-25T15:03:27.749 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2026-02-25T15:03:27.749 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (3164 ms) 2026-02-25T15:03:27.749 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2026-02-25T15:03:27.749 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (4328 ms) 2026-02-25T15:03:27.749 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2026-02-25T15:03:27.749 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (5863 ms) 2026-02-25T15:03:27.750 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2026-02-25T15:03:27.750 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (14098 ms) 2026-02-25T15:03:27.750 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2026-02-25T15:03:27.750 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (30308 ms) 2026-02-25T15:03:27.750 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Cancel 2026-02-25T15:03:27.750 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ OK ] NeoRadosWriteOps.Cancel (7222 ms) 2026-02-25T15:03:27.750 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [----------] 8 tests from NeoRadosWriteOps (77085 ms total) 2026-02-25T15:03:27.750 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: 2026-02-25T15:03:27.750 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [----------] Global test environment tear-down 2026-02-25T15:03:27.750 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [==========] 8 tests from 1 test suite ran. (77085 ms total) 2026-02-25T15:03:27.750 INFO:tasks.workunit.client.0.trial189.stdout: write_operations: [ PASSED ] 8 tests. 2026-02-25T15:03:27.804 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: Running main() from gmock_main.cc 2026-02-25T15:03:27.804 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2026-02-25T15:03:27.804 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [----------] Global test environment set-up. 2026-02-25T15:03:27.804 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [----------] 9 tests from LibRadosPools 2026-02-25T15:03:27.804 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2026-02-25T15:03:27.805 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ OK ] LibRadosPools.PoolList (3423 ms) 2026-02-25T15:03:27.805 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2026-02-25T15:03:27.805 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (5559 ms) 2026-02-25T15:03:27.805 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2026-02-25T15:03:27.805 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (3192 ms) 2026-02-25T15:03:27.805 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2026-02-25T15:03:27.805 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3178 ms) 2026-02-25T15:03:27.805 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2026-02-25T15:03:27.805 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3236 ms) 2026-02-25T15:03:27.805 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2026-02-25T15:03:27.805 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (7199 ms) 2026-02-25T15:03:27.805 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2026-02-25T15:03:27.805 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (15879 ms) 2026-02-25T15:03:27.806 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2026-02-25T15:03:27.806 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (16445 ms) 2026-02-25T15:03:27.806 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2026-02-25T15:03:27.806 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (19106 ms) 2026-02-25T15:03:27.806 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [----------] 9 tests from LibRadosPools (77221 ms total) 2026-02-25T15:03:27.806 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: 2026-02-25T15:03:27.806 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [----------] Global test environment tear-down 2026-02-25T15:03:27.806 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (77222 ms total) 2026-02-25T15:03:27.806 INFO:tasks.workunit.client.0.trial189.stdout: api_pool: [ PASSED ] 9 tests. 2026-02-25T15:03:31.913 INFO:tasks.workunit.client.0.trial189.stdout: misc: Running main() from gmock_main.cc 2026-02-25T15:03:31.913 INFO:tasks.workunit.client.0.trial189.stdout: misc: [==========] Running 12 tests from 1 test suite. 2026-02-25T15:03:31.913 INFO:tasks.workunit.client.0.trial189.stdout: misc: [----------] Global test environment set-up. 2026-02-25T15:03:31.914 INFO:tasks.workunit.client.0.trial189.stdout: misc: [----------] 12 tests from NeoRadosMisc 2026-02-25T15:03:31.914 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ RUN ] NeoRadosMisc.Version 2026-02-25T15:03:31.914 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ OK ] NeoRadosMisc.Version (2185 ms) 2026-02-25T15:03:31.914 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2026-02-25T15:03:31.914 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (3046 ms) 2026-02-25T15:03:31.914 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2026-02-25T15:03:31.914 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ OK ] NeoRadosMisc.LongName (6876 ms) 2026-02-25T15:03:31.914 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2026-02-25T15:03:31.914 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (3170 ms) 2026-02-25T15:03:31.914 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2026-02-25T15:03:31.914 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (3276 ms) 2026-02-25T15:03:31.914 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2026-02-25T15:03:31.915 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (2440 ms) 2026-02-25T15:03:31.915 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2026-02-25T15:03:31.915 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ OK ] NeoRadosMisc.Exec (5939 ms) 2026-02-25T15:03:31.915 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2026-02-25T15:03:31.915 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (12633 ms) 2026-02-25T15:03:31.915 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2026-02-25T15:03:31.915 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (13170 ms) 2026-02-25T15:03:31.915 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2026-02-25T15:03:31.915 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ OK ] NeoRadosMisc.BigObject (11704 ms) 2026-02-25T15:03:31.915 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2026-02-25T15:03:31.915 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (8752 ms) 2026-02-25T15:03:31.915 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2026-02-25T15:03:31.916 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (8067 ms) 2026-02-25T15:03:31.916 INFO:tasks.workunit.client.0.trial189.stdout: misc: [----------] 12 tests from NeoRadosMisc (81263 ms total) 2026-02-25T15:03:31.916 INFO:tasks.workunit.client.0.trial189.stdout: misc: 2026-02-25T15:03:31.916 INFO:tasks.workunit.client.0.trial189.stdout: misc: [----------] Global test environment tear-down 2026-02-25T15:03:31.916 INFO:tasks.workunit.client.0.trial189.stdout: misc: [==========] 12 tests from 1 test suite ran. (81263 ms total) 2026-02-25T15:03:31.916 INFO:tasks.workunit.client.0.trial189.stdout: misc: [ PASSED ] 12 tests. 2026-02-25T15:03:35.096 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2026-02-25T15:03:35.096 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [----------] Global test environment set-up. 2026-02-25T15:03:35.096 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2026-02-25T15:03:35.096 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: 2026-02-25T15:02:10.825+0000 7f8f4d9f0880 -1 failed for service _ceph-mon._tcp 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: 2026-02-25T15:02:10.825+0000 7f8f4d9f0880 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (246 ms) 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (5002 ms) 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (5248 ms total) 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: started 0x55cd28d95fb0 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: started 0x55cd28f7b520 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: started 2 aios 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: waiting 0x55cd28d95fb0 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: waiting 0x55cd28f7b520 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: done. 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (9082 ms) 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (9082 ms total) 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ OK ] LibRadosMisc.Exec (750 ms) 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (93 ms) 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (36 ms) 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ OK ] LibRadosMisc.Applications (4811 ms) 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2026-02-25T15:03:35.097 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2026-02-25T15:03:35.098 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2026-02-25T15:03:35.098 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (61437 ms) 2026-02-25T15:03:35.098 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [----------] 8 tests from LibRadosMisc (67130 ms total) 2026-02-25T15:03:35.098 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: 2026-02-25T15:03:35.098 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [----------] Global test environment tear-down 2026-02-25T15:03:35.098 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (84500 ms total) 2026-02-25T15:03:35.098 INFO:tasks.workunit.client.0.trial189.stdout: api_misc: [ PASSED ] 12 tests. 2026-02-25T15:03:44.107 INFO:tasks.workunit.client.0.trial189.stdout: io: Running main() from gmock_main.cc 2026-02-25T15:03:44.107 INFO:tasks.workunit.client.0.trial189.stdout: io: [==========] Running 14 tests from 1 test suite. 2026-02-25T15:03:44.107 INFO:tasks.workunit.client.0.trial189.stdout: io: [----------] Global test environment set-up. 2026-02-25T15:03:44.107 INFO:tasks.workunit.client.0.trial189.stdout: io: [----------] 14 tests from NeoRadosIo 2026-02-25T15:03:44.107 INFO:tasks.workunit.client.0.trial189.stdout: io: [ RUN ] NeoRadosIo.Limits 2026-02-25T15:03:44.107 INFO:tasks.workunit.client.0.trial189.stdout: io: [ OK ] NeoRadosIo.Limits (7900 ms) 2026-02-25T15:03:44.107 INFO:tasks.workunit.client.0.trial189.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2026-02-25T15:03:44.108 INFO:tasks.workunit.client.0.trial189.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (4249 ms) 2026-02-25T15:03:44.108 INFO:tasks.workunit.client.0.trial189.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2026-02-25T15:03:44.108 INFO:tasks.workunit.client.0.trial189.stdout: io: [ OK ] NeoRadosIo.ReadOp (3158 ms) 2026-02-25T15:03:44.108 INFO:tasks.workunit.client.0.trial189.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2026-02-25T15:03:44.108 INFO:tasks.workunit.client.0.trial189.stdout: io: [ OK ] NeoRadosIo.SparseRead (3225 ms) 2026-02-25T15:03:44.108 INFO:tasks.workunit.client.0.trial189.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2026-02-25T15:03:44.108 INFO:tasks.workunit.client.0.trial189.stdout: io: [ OK ] NeoRadosIo.RoundTrip (3519 ms) 2026-02-25T15:03:44.108 INFO:tasks.workunit.client.0.trial189.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2026-02-25T15:03:44.108 INFO:tasks.workunit.client.0.trial189.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (9149 ms) 2026-02-25T15:03:44.108 INFO:tasks.workunit.client.0.trial189.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2026-02-25T15:03:44.108 INFO:tasks.workunit.client.0.trial189.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (11991 ms) 2026-02-25T15:03:44.108 INFO:tasks.workunit.client.0.trial189.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2026-02-25T15:03:44.109 INFO:tasks.workunit.client.0.trial189.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (10871 ms) 2026-02-25T15:03:44.109 INFO:tasks.workunit.client.0.trial189.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2026-02-25T15:03:44.109 INFO:tasks.workunit.client.0.trial189.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (13609 ms) 2026-02-25T15:03:44.109 INFO:tasks.workunit.client.0.trial189.stdout: io: [ RUN ] NeoRadosIo.Trunc 2026-02-25T15:03:44.109 INFO:tasks.workunit.client.0.trial189.stdout: io: [ OK ] NeoRadosIo.Trunc (9487 ms) 2026-02-25T15:03:44.109 INFO:tasks.workunit.client.0.trial189.stdout: io: [ RUN ] NeoRadosIo.Remove 2026-02-25T15:03:44.109 INFO:tasks.workunit.client.0.trial189.stdout: io: [ OK ] NeoRadosIo.Remove (7311 ms) 2026-02-25T15:03:44.109 INFO:tasks.workunit.client.0.trial189.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2026-02-25T15:03:44.109 INFO:tasks.workunit.client.0.trial189.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (2866 ms) 2026-02-25T15:03:44.109 INFO:tasks.workunit.client.0.trial189.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2026-02-25T15:03:44.109 INFO:tasks.workunit.client.0.trial189.stdout: io: [ OK ] NeoRadosIo.RmXattr (2953 ms) 2026-02-25T15:03:44.109 INFO:tasks.workunit.client.0.trial189.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2026-02-25T15:03:44.110 INFO:tasks.workunit.client.0.trial189.stdout: io: [ OK ] NeoRadosIo.GetXattrs (3170 ms) 2026-02-25T15:03:44.110 INFO:tasks.workunit.client.0.trial189.stdout: io: [----------] 14 tests from NeoRadosIo (93466 ms total) 2026-02-25T15:03:44.110 INFO:tasks.workunit.client.0.trial189.stdout: io: 2026-02-25T15:03:44.110 INFO:tasks.workunit.client.0.trial189.stdout: io: [----------] Global test environment tear-down 2026-02-25T15:03:44.110 INFO:tasks.workunit.client.0.trial189.stdout: io: [==========] 14 tests from 1 test suite ran. (93466 ms total) 2026-02-25T15:03:44.110 INFO:tasks.workunit.client.0.trial189.stdout: io: [ PASSED ] 14 tests. 2026-02-25T15:03:45.948 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: Running main() from gmock_main.cc 2026-02-25T15:03:45.948 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [==========] Running 15 tests from 1 test suite. 2026-02-25T15:03:45.948 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [----------] Global test environment set-up. 2026-02-25T15:03:45.948 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [----------] 15 tests from NeoRadosReadOps 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (8990 ms) 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (3118 ms) 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (3180 ms) 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (3244 ms) 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (3507 ms) 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (9331 ms) 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (11811 ms) 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (11120 ms) 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (13384 ms) 2026-02-25T15:03:45.949 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (9461 ms) 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (6260 ms) 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (2868 ms) 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (3198 ms) 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (2931 ms) 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ RUN ] NeoRadosReadOps.Cancel 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ OK ] NeoRadosReadOps.Cancel (2882 ms) 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [----------] 15 tests from NeoRadosReadOps (95292 ms total) 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [----------] Global test environment tear-down 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [==========] 15 tests from 1 test suite ran. (95292 ms total) 2026-02-25T15:03:45.950 INFO:tasks.workunit.client.0.trial189.stdout: read_operations: [ PASSED ] 15 tests. 2026-02-25T15:03:50.926 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:03:04.433 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:50.929 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:03:04.436 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:50.957 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:03:04.427 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:50.958 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:03:04.460 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:50.958 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:03:04.428 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:50.959 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:03:04.461 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:50.961 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:03:04.468 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:50.990 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:03:04.459 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:50.991 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:03:04.493 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:03:56.026 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: Running main() from gmock_main.cc 2026-02-25T15:03:56.026 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2026-02-25T15:03:56.027 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [----------] Global test environment set-up. 2026-02-25T15:03:56.027 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2026-02-25T15:03:56.027 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2026-02-25T15:03:56.027 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (7831 ms) 2026-02-25T15:03:56.027 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2026-02-25T15:03:56.027 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (2089 ms) 2026-02-25T15:03:56.027 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2026-02-25T15:03:56.027 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2153 ms) 2026-02-25T15:03:56.027 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2026-02-25T15:03:56.027 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2111 ms) 2026-02-25T15:03:56.027 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (14185 ms total) 2026-02-25T15:03:56.028 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: 2026-02-25T15:03:56.028 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2026-02-25T15:03:56.028 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2026-02-25T15:03:56.028 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (11837 ms) 2026-02-25T15:03:56.028 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2026-02-25T15:03:56.028 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (14274 ms) 2026-02-25T15:03:56.028 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2026-02-25T15:03:56.028 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (13999 ms) 2026-02-25T15:03:56.028 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (40111 ms total) 2026-02-25T15:03:56.028 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: 2026-02-25T15:03:56.028 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2026-02-25T15:03:56.028 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2026-02-25T15:03:56.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (5012 ms) 2026-02-25T15:03:56.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2026-02-25T15:03:56.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (1917 ms) 2026-02-25T15:03:56.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2026-02-25T15:03:56.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (1852 ms) 2026-02-25T15:03:56.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2026-02-25T15:03:56.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2124 ms) 2026-02-25T15:03:56.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (10906 ms total) 2026-02-25T15:03:56.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: 2026-02-25T15:03:56.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2026-02-25T15:03:56.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2026-02-25T15:03:56.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4133 ms) 2026-02-25T15:03:56.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2026-02-25T15:03:56.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4338 ms) 2026-02-25T15:03:56.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8472 ms total) 2026-02-25T15:03:56.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: 2026-02-25T15:03:56.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [----------] Global test environment tear-down 2026-02-25T15:03:56.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (105442 ms total) 2026-02-25T15:03:56.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots: [ PASSED ] 13 tests. 2026-02-25T15:03:56.031 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:03:56.032 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49773 2026-02-25T15:03:56.032 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49773 2026-02-25T15:03:56.032 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:03:56.032 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49525 2026-02-25T15:03:56.032 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49525 2026-02-25T15:03:56.032 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:03:56.032 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49413 2026-02-25T15:03:56.032 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49413 2026-02-25T15:03:56.032 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:03:56.032 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49270 2026-02-25T15:03:56.032 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49270 2026-02-25T15:03:56.033 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:03:56.033 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49468 2026-02-25T15:03:56.033 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49468 2026-02-25T15:03:56.033 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:03:56.033 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49877 2026-02-25T15:03:56.033 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49877 2026-02-25T15:03:56.033 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:03:56.033 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49914 2026-02-25T15:03:56.033 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49914 2026-02-25T15:03:56.033 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:03:56.033 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49292 2026-02-25T15:03:56.033 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49292 2026-02-25T15:04:06.028 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: Running main() from gmock_main.cc 2026-02-25T15:04:06.028 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2026-02-25T15:04:06.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2026-02-25T15:04:06.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2026-02-25T15:04:06.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: seed 49501 2026-02-25T15:04:06.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2026-02-25T15:04:06.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (7872 ms) 2026-02-25T15:04:06.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2026-02-25T15:04:06.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (3159 ms) 2026-02-25T15:04:06.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2026-02-25T15:04:06.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (2076 ms) 2026-02-25T15:04:06.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2026-02-25T15:04:06.029 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2177 ms) 2026-02-25T15:04:06.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2026-02-25T15:04:06.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (4585 ms) 2026-02-25T15:04:06.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (19872 ms total) 2026-02-25T15:04:06.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: 2026-02-25T15:04:06.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2026-02-25T15:04:06.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2026-02-25T15:04:06.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (12049 ms) 2026-02-25T15:04:06.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2026-02-25T15:04:06.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (10267 ms) 2026-02-25T15:04:06.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2026-02-25T15:04:06.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (15843 ms) 2026-02-25T15:04:06.030 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2026-02-25T15:04:06.031 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (6748 ms) 2026-02-25T15:04:06.031 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2026-02-25T15:04:06.031 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2922 ms) 2026-02-25T15:04:06.031 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2026-02-25T15:04:06.031 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: /ceph/rpmbuild/BUILD/ceph-20.2.0-676-gc824a7fa/src/test/librados/snapshots_cxx.cc:460: Skipped 2026-02-25T15:04:06.031 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: 2026-02-25T15:04:06.031 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: 2026-02-25T15:04:06.031 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2026-02-25T15:04:06.031 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2026-02-25T15:04:06.031 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_trial189-49501-7 2026-02-25T15:04:06.031 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: waiting for snaps to purge 2026-02-25T15:04:06.031 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (18314 ms) 2026-02-25T15:04:06.031 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (66147 ms total) 2026-02-25T15:04:06.032 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: 2026-02-25T15:04:06.032 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2026-02-25T15:04:06.032 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2026-02-25T15:04:06.032 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (2 ms) 2026-02-25T15:04:06.032 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2026-02-25T15:04:06.032 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (4828 ms) 2026-02-25T15:04:06.032 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (4830 ms total) 2026-02-25T15:04:06.032 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: 2026-02-25T15:04:06.032 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2026-02-25T15:04:06.032 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2026-02-25T15:04:06.032 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3147 ms) 2026-02-25T15:04:06.032 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2026-02-25T15:04:06.033 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (2012 ms) 2026-02-25T15:04:06.033 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2026-02-25T15:04:10.978 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:03:24.485 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:10.980 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:03:24.487 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:11.004 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:03:24.511 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:11.006 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:03:24.475 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:11.006 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:03:24.509 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:11.008 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:03:24.477 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:11.014 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:03:24.517 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:11.034 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:03:24.503 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:11.039 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:03:24.542 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:11.298 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ snapshots: Running main() from gmock_main.cc 2026-02-25T15:04:11.298 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2026-02-25T15:04:11.299 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [----------] Global test environment set-up. 2026-02-25T15:04:11.299 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2026-02-25T15:04:11.299 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2026-02-25T15:04:11.299 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (9985 ms) 2026-02-25T15:04:11.299 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2026-02-25T15:04:11.299 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (5286 ms) 2026-02-25T15:04:11.299 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2026-02-25T15:04:11.299 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (4367 ms) 2026-02-25T15:04:11.299 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2026-02-25T15:04:11.299 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (13469 ms) 2026-02-25T15:04:11.299 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2026-02-25T15:04:11.300 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (21175 ms) 2026-02-25T15:04:11.300 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (54285 ms total) 2026-02-25T15:04:11.300 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: 2026-02-25T15:04:11.300 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2026-02-25T15:04:11.300 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2026-02-25T15:04:11.300 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (18800 ms) 2026-02-25T15:04:11.300 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2026-02-25T15:04:11.300 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (10335 ms) 2026-02-25T15:04:11.300 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2026-02-25T15:04:11.300 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (7936 ms) 2026-02-25T15:04:11.300 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2026-02-25T15:04:11.300 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (6041 ms) 2026-02-25T15:04:11.301 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2026-02-25T15:04:11.301 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4272 ms) 2026-02-25T15:04:11.301 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2026-02-25T15:04:11.301 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnaptrial189-50235-11. 2026-02-25T15:04:11.301 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: Waiting for snaps to purge. 2026-02-25T15:04:11.301 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (18966 ms) 2026-02-25T15:04:11.301 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (66354 ms total) 2026-02-25T15:04:11.301 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: 2026-02-25T15:04:11.301 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [----------] Global test environment tear-down 2026-02-25T15:04:11.301 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (120639 ms total) 2026-02-25T15:04:11.301 INFO:tasks.workunit.client.0.trial189.stdout: snapshots: [ PASSED ] 11 tests. 2026-02-25T15:04:12.336 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2026-02-25T15:04:12.336 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] Global test environment set-up. 2026-02-25T15:04:12.336 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2026-02-25T15:04:12.336 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2026-02-25T15:04:12.337 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2026-02-25T15:04:12.337 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2026-02-25T15:04:12.337 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: 2026-02-25T15:04:12.337 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2026-02-25T15:04:12.337 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: seed 49441 2026-02-25T15:04:12.337 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2026-02-25T15:04:12.337 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (3 ms) 2026-02-25T15:04:12.337 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2026-02-25T15:04:12.337 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (2946 ms) 2026-02-25T15:04:12.337 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2026-02-25T15:04:12.337 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (618 ms) 2026-02-25T15:04:12.337 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2026-02-25T15:04:12.338 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (106 ms) 2026-02-25T15:04:12.338 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2026-02-25T15:04:12.338 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (116 ms) 2026-02-25T15:04:12.338 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2026-02-25T15:04:12.338 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (19 ms) 2026-02-25T15:04:12.338 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2026-02-25T15:04:12.338 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (24 ms) 2026-02-25T15:04:12.338 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2026-02-25T15:04:12.338 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (84 ms) 2026-02-25T15:04:12.338 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2026-02-25T15:04:12.338 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (29 ms) 2026-02-25T15:04:12.338 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2026-02-25T15:04:12.339 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (103 ms) 2026-02-25T15:04:12.339 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2026-02-25T15:04:12.339 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (29 ms) 2026-02-25T15:04:12.339 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2026-02-25T15:04:12.339 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (116 ms) 2026-02-25T15:04:12.339 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2026-02-25T15:04:12.339 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (105 ms) 2026-02-25T15:04:12.339 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2026-02-25T15:04:12.339 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: osd_max_attr_size = 0 2026-02-25T15:04:12.339 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2026-02-25T15:04:12.339 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (35480 ms) 2026-02-25T15:04:12.339 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2026-02-25T15:04:12.339 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (5225 ms) 2026-02-25T15:04:12.340 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2026-02-25T15:04:12.340 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: waiting for initial deep scrubs... 2026-02-25T15:04:12.340 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: done waiting, doing copies 2026-02-25T15:04:12.340 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: waiting for final deep scrubs... 2026-02-25T15:04:12.340 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: done waiting 2026-02-25T15:04:12.340 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (68000 ms) 2026-02-25T15:04:12.340 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2026-02-25T15:04:12.340 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (43 ms) 2026-02-25T15:04:12.340 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2026-02-25T15:04:12.340 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (22 ms) 2026-02-25T15:04:12.340 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2026-02-25T15:04:12.340 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (5380 ms) 2026-02-25T15:04:12.340 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2026-02-25T15:04:12.341 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2026-02-25T15:04:12.341 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2026-02-25T15:04:12.341 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2026-02-25T15:04:12.341 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2026-02-25T15:04:12.341 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2026-02-25T15:04:25.376 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] 22 tests f api_aio: Running main() from gmock_main.cc 2026-02-25T15:04:25.376 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [==========] Running 45 tests from 2 test suites. 2026-02-25T15:04:25.376 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [----------] Global test environment set-up. 2026-02-25T15:04:25.377 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [----------] 29 tests from LibRadosAio 2026-02-25T15:04:25.377 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2026-02-25T15:04:25.377 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.TooBig (3447 ms) 2026-02-25T15:04:25.377 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2026-02-25T15:04:25.377 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (7759 ms) 2026-02-25T15:04:25.377 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2026-02-25T15:04:25.377 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (3152 ms) 2026-02-25T15:04:25.377 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2026-02-25T15:04:25.377 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3243 ms) 2026-02-25T15:04:25.377 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2026-02-25T15:04:25.377 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3484 ms) 2026-02-25T15:04:25.378 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2026-02-25T15:04:25.378 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (7934 ms) 2026-02-25T15:04:25.378 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2026-02-25T15:04:25.378 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (10549 ms) 2026-02-25T15:04:25.378 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2026-02-25T15:04:25.378 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (9355 ms) 2026-02-25T15:04:25.378 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2026-02-25T15:04:25.378 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (10893 ms) 2026-02-25T15:04:25.378 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2026-02-25T15:04:25.378 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (15156 ms) 2026-02-25T15:04:25.378 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2026-02-25T15:04:25.378 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (8571 ms) 2026-02-25T15:04:25.378 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2026-02-25T15:04:25.379 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (2831 ms) 2026-02-25T15:04:25.379 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2026-02-25T15:04:25.379 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3197 ms) 2026-02-25T15:04:25.379 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2026-02-25T15:04:25.379 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2942 ms) 2026-02-25T15:04:25.379 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2026-02-25T15:04:25.379 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.Flush (2877 ms) 2026-02-25T15:04:25.379 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2026-02-25T15:04:25.379 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3112 ms) 2026-02-25T15:04:25.379 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2026-02-25T15:04:25.379 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3271 ms) 2026-02-25T15:04:25.379 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2026-02-25T15:04:25.379 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3207 ms) 2026-02-25T15:04:25.380 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2026-02-25T15:04:25.380 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (2526 ms) 2026-02-25T15:04:25.380 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2026-02-25T15:04:25.380 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3210 ms) 2026-02-25T15:04:25.380 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2026-02-25T15:04:25.380 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (3063 ms) 2026-02-25T15:04:25.380 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2026-02-25T15:04:25.380 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (2758 ms) 2026-02-25T15:04:25.380 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2026-02-25T15:04:25.380 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3168 ms) 2026-02-25T15:04:25.380 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2026-02-25T15:04:25.380 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3059 ms) 2026-02-25T15:04:25.380 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2026-02-25T15:04:25.381 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (2697 ms) 2026-02-25T15:04:25.381 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2026-02-25T15:04:25.381 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3097 ms) 2026-02-25T15:04:25.381 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.CancelBeforeSubmit 2026-02-25T15:04:25.381 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.CancelBeforeSubmit (3193 ms) 2026-02-25T15:04:25.381 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.CancelBeforeComplete 2026-02-25T15:04:29.123 INFO:tasks.workunit.client.0.trial189.stdout: OK ] LibRadosSnapshotsECPP.RollbackPP (1719 ms) 2026-02-25T15:04:29.123 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2026-02-25T15:04:29.124 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2168 ms) 2026-02-25T15:04:29.124 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (9047 ms total) 2026-02-25T15:04:29.124 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: 2026-02-25T15:04:29.124 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2026-02-25T15:04:29.124 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2026-02-25T15:04:29.124 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (3883 ms) 2026-02-25T15:04:29.124 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2026-02-25T15:04:29.124 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (4110 ms) 2026-02-25T15:04:29.124 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2026-02-25T15:04:29.124 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3708 ms) 2026-02-25T15:04:29.124 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11702 ms total) 2026-02-25T15:04:29.125 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: 2026-02-25T15:04:29.125 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2026-02-25T15:04:29.125 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (138535 ms total) 2026-02-25T15:04:29.125 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2026-02-25T15:04:29.125 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2026-02-25T15:04:29.125 INFO:tasks.workunit.client.0.trial189.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2026-02-25T15:04:31.027 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:03:44.534 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:31.028 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:03:44.535 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:31.055 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:03:44.558 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:31.056 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:03:44.559 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:31.057 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:03:44.526 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:31.057 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:03:44.527 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:31.070 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:03:44.577 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:31.083 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:03:44.552 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:31.083 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:03:44.586 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:36.056 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: Running main() from gmock_main.cc 2026-02-25T15:04:36.056 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [==========] Running 63 tests from 5 test suites. 2026-02-25T15:04:36.056 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [----------] Global test environment set-up. 2026-02-25T15:04:36.056 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2026-02-25T15:04:36.057 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2026-02-25T15:04:36.057 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (3440 ms) 2026-02-25T15:04:36.057 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2026-02-25T15:04:36.057 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (25489 ms) 2026-02-25T15:04:36.057 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2026-02-25T15:04:36.057 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (20006 ms) 2026-02-25T15:04:36.057 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2026-02-25T15:04:36.057 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (9271 ms) 2026-02-25T15:04:36.057 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2026-02-25T15:04:36.057 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (11571 ms) 2026-02-25T15:04:36.057 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2026-02-25T15:04:36.057 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (9888 ms) 2026-02-25T15:04:36.058 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2026-02-25T15:04:36.058 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (5704 ms) 2026-02-25T15:04:36.058 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2026-02-25T15:04:36.058 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3143 ms) 2026-02-25T15:04:36.058 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2026-02-25T15:04:36.058 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2941 ms) 2026-02-25T15:04:36.058 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2026-02-25T15:04:36.058 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3093 ms) 2026-02-25T15:04:36.058 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2026-02-25T15:04:36.058 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2947 ms) 2026-02-25T15:04:36.058 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2026-02-25T15:04:36.058 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3099 ms) 2026-02-25T15:04:36.058 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2026-02-25T15:04:36.059 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3304 ms) 2026-02-25T15:04:36.059 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2026-02-25T15:04:36.059 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (2521 ms) 2026-02-25T15:04:36.059 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2026-02-25T15:04:36.059 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3220 ms) 2026-02-25T15:04:36.059 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2026-02-25T15:04:36.059 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3103 ms) 2026-02-25T15:04:36.059 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2026-02-25T15:04:36.059 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (2716 ms) 2026-02-25T15:04:36.059 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2026-02-25T15:04:36.059 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3250 ms) 2026-02-25T15:04:36.059 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2026-02-25T15:04:36.059 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3032 ms) 2026-02-25T15:04:36.060 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2026-02-25T15:04:36.060 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3126 ms) 2026-02-25T15:04:36.060 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2026-02-25T15:04:36.060 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (2685 ms) 2026-02-25T15:04:36.060 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2026-02-25T15:04:36.060 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3128 ms) 2026-02-25T15:04:36.060 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2026-02-25T15:04:36.060 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3113 ms) 2026-02-25T15:04:36.060 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2026-02-25T15:04:36.060 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (2729 ms) 2026-02-25T15:04:36.060 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2026-02-25T15:04:36.060 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3094 ms) 2026-02-25T15:04:36.060 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2026-02-25T15:04:36.061 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3048 ms) 2026-02-25T15:04:36.061 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2026-02-25T15:04:39.224 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosArom LibRadosMiscPP (118459 ms total) 2026-02-25T15:04:39.225 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: 2026-02-25T15:04:39.225 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2026-02-25T15:04:39.225 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2026-02-25T15:04:39.225 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (1450 ms) 2026-02-25T15:04:39.225 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (1450 ms total) 2026-02-25T15:04:39.226 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: 2026-02-25T15:04:39.226 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0,Checksummer::xxhash32,ceph_le > 2026-02-25T15:04:39.226 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2026-02-25T15:04:39.226 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (284 ms) 2026-02-25T15:04:39.226 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2026-02-25T15:04:39.226 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (25 ms) 2026-02-25T15:04:39.226 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (309 ms total) 2026-02-25T15:04:39.227 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: 2026-02-25T15:04:39.227 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1,Checksummer::xxhash64,ceph_le > 2026-02-25T15:04:39.227 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2026-02-25T15:04:39.227 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (168 ms) 2026-02-25T15:04:39.227 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2026-02-25T15:04:39.227 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (30 ms) 2026-02-25T15:04:39.228 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (199 ms total) 2026-02-25T15:04:39.228 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: 2026-02-25T15:04:39.228 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2,Checksummer::crc32c,ceph_le > 2026-02-25T15:04:39.228 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2026-02-25T15:04:39.228 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (106 ms) 2026-02-25T15:04:39.228 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2026-02-25T15:04:39.229 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (44 ms) 2026-02-25T15:04:39.229 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (150 ms total) 2026-02-25T15:04:39.229 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: 2026-02-25T15:04:39.229 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2026-02-25T15:04:39.229 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2026-02-25T15:04:39.229 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (980 ms) 2026-02-25T15:04:39.229 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (980 ms total) 2026-02-25T15:04:39.230 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: 2026-02-25T15:04:39.230 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [----------] Global test environment tear-down 2026-02-25T15:04:39.230 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (148634 ms total) 2026-02-25T15:04:39.230 INFO:tasks.workunit.client.0.trial189.stdout: api_misc_pp: [ PASSED ] 31 tests. 2026-02-25T15:04:39.232 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:04:39.232 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49794 2026-02-25T15:04:39.232 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49794 2026-02-25T15:04:39.233 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:04:39.233 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49167 2026-02-25T15:04:39.233 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49167 2026-02-25T15:04:39.233 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:04:39.233 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49230 2026-02-25T15:04:39.233 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49230 2026-02-25T15:04:39.233 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:04:39.234 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49705 2026-02-25T15:04:39.234 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49705 2026-02-25T15:04:39.234 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:04:39.234 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49157 2026-02-25T15:04:39.234 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49157 2026-02-25T15:04:51.065 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:04:04.572 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:51.073 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:04:04.580 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:51.098 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:04:04.605 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:51.099 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:04:04.602 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:51.100 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:04:04.603 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:51.101 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:04:04.570 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:51.102 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:04:04.571 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:51.125 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:04:04.594 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:04:51.126 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:04:04.628 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:31.162 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:04:44.669 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:31.164 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:04:44.671 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:31.191 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:04:44.693 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:31.192 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:04:44.694 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:31.192 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:04:44.661 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:31.193 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:04:44.662 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:31.205 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:04:44.713 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:31.231 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:04:44.700 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:31.232 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:04:44.734 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:34.320 INFO:tasks.workunit.client.0.trial189.stdout:cursor()=8:e960b815:::9:head expected=8:e960b815:::9:head 2026-02-25T15:05:34.320 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:e960b815:::9:head -> 9 2026-02-25T15:05:34.320 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=9 expected=9 2026-02-25T15:05:34.321 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:de5d7c5f:::12:head 2026-02-25T15:05:34.321 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : cursor()=8:de5d7c5f:::12:head expected=8:de5d7c5f:::12:head 2026-02-25T15:05:34.321 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:de5d7c5f:::12:head -> 12 2026-02-25T15:05:34.321 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=12 expected=12 2026-02-25T15:05:34.321 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:5c6b0b28:::7:head 2026-02-25T15:05:34.321 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : cursor()=8:5c6b0b28:::7:head expected=8:5c6b0b28:::7:head 2026-02-25T15:05:34.321 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:5c6b0b28:::7:head -> 7 2026-02-25T15:05:34.321 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=7 expected=7 2026-02-25T15:05:34.321 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:566253c9:::13:head 2026-02-25T15:05:34.321 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : cursor()=8:566253c9:::13:head expected=8:566253c9:::13:head 2026-02-25T15:05:34.321 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:566253c9:::13:head -> 13 2026-02-25T15:05:34.321 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=13 expected=13 2026-02-25T15:05:34.322 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:52ea6a34:::10:head 2026-02-25T15:05:34.322 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : cursor()=8:52ea6a34:::10:head expected=8:52ea6a34:::10:head 2026-02-25T15:05:34.322 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:52ea6a34:::10:head -> 10 2026-02-25T15:05:34.322 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=10 expected=10 2026-02-25T15:05:34.322 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:6cac518f:::0:head 2026-02-25T15:05:34.322 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : cursor()=8:6cac518f:::0:head expected=8:6cac518f:::0:head 2026-02-25T15:05:34.322 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:6cac518f:::0:head -> 0 2026-02-25T15:05:34.322 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=0 expected=0 2026-02-25T15:05:34.322 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:62a1935d:::14:head 2026-02-25T15:05:34.322 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : cursor()=8:62a1935d:::14:head expected=8:62a1935d:::14:head 2026-02-25T15:05:34.322 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:62a1935d:::14:head -> 14 2026-02-25T15:05:34.322 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=14 expected=14 2026-02-25T15:05:34.322 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : seek to 8:863748b0:::15:head 2026-02-25T15:05:34.323 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : cursor()=8:863748b0:::15:head expected=8:863748b0:::15:head 2026-02-25T15:05:34.323 INFO:tasks.workunit.client.0.trial189.stdout: api_list: > 8:863748b0:::15:head -> 15 2026-02-25T15:05:34.323 INFO:tasks.workunit.client.0.trial189.stdout: api_list: : entry=15 expected=15 2026-02-25T15:05:34.323 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (9919 ms) 2026-02-25T15:05:34.323 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2026-02-25T15:05:34.323 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (132500 ms) 2026-02-25T15:05:34.323 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2026-02-25T15:05:34.323 INFO:tasks.workunit.client.0.trial189.stdout: api_list: split 0/5 -> MIN 8:33333333::::head 2026-02-25T15:05:34.323 INFO:tasks.workunit.client.0.trial189.stdout: api_list: split 1/5 -> 8:33333333::::head 8:66666666::::head 2026-02-25T15:05:34.323 INFO:tasks.workunit.client.0.trial189.stdout: api_list: split 2/5 -> 8:66666666::::head 8:99999999::::head 2026-02-25T15:05:34.323 INFO:tasks.workunit.client.0.trial189.stdout: api_list: split 3/5 -> 8:99999999::::head 8:cccccccc::::head 2026-02-25T15:05:34.323 INFO:tasks.workunit.client.0.trial189.stdout: api_list: split 4/5 -> 8:cccccccc::::head MAX 2026-02-25T15:05:34.324 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (42648 ms) 2026-02-25T15:05:34.324 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [----------] 7 tests from LibRadosList (195948 ms total) 2026-02-25T15:05:34.324 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 2026-02-25T15:05:34.324 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [----------] 3 tests from LibRadosListEC 2026-02-25T15:05:34.324 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2026-02-25T15:05:34.324 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1220 ms) 2026-02-25T15:05:34.324 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2026-02-25T15:05:34.324 INFO:tasks.workunit.client.0.trial189.stdout: api_list: myset foo1,foo2,foo3 2026-02-25T15:05:34.324 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo1 2026-02-25T15:05:34.324 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo2 2026-02-25T15:05:34.324 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo3 2026-02-25T15:05:34.324 INFO:tasks.workunit.client.0.trial189.stdout: api_list: myset foo1,foo4,foo5 2026-02-25T15:05:34.324 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo4 2026-02-25T15:05:34.325 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo5 2026-02-25T15:05:34.325 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo1 2026-02-25T15:05:34.325 INFO:tasks.workunit.client.0.trial189.stdout: api_list: myset foo6,foo7 2026-02-25T15:05:34.325 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo7 2026-02-25T15:05:34.325 INFO:tasks.workunit.client.0.trial189.stdout: api_list: foo6 2026-02-25T15:05:34.325 INFO:tasks.workunit.client.0.trial189.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2026-02-25T15:05:34.325 INFO:tasks.workunit.client.0.trial189.stdout: api_list: ns1:foo4 2026-02-25T15:05:34.325 INFO:tasks.workunit.client.0.trial189.stdout: api_list: ns1:foo5 2026-02-25T15:05:34.325 INFO:tasks.workunit.client.0.trial189.stdout: api_list: ns2:foo7 2026-02-25T15:05:34.325 INFO:tasks.workunit.client.0.trial189.stdout: api_list: ns2:foo6 2026-02-25T15:05:34.325 INFO:tasks.workunit.client.0.trial189.stdout: api_list: ns1:foo1 2026-02-25T15:05:34.325 INFO:tasks.workunit.client.0.trial189.stdout: api_list: :foo1 2026-02-25T15:05:34.326 INFO:tasks.workunit.client.0.trial189.stdout: api_list: :foo2 2026-02-25T15:05:34.326 INFO:tasks.workunit.client.0.trial189.stdout: api_list: :foo3 2026-02-25T15:05:34.326 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (425 ms) 2026-02-25T15:05:34.326 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2026-02-25T15:05:34.326 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 1 0 2026-02-25T15:05:34.326 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 10 0 2026-02-25T15:05:34.326 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 13 0 2026-02-25T15:05:40.352 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 7 0 2026-02-25T15:05:40.352 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 14 0 2026-02-25T15:05:40.352 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 0 0 2026-02-25T15:05:40.352 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 15 0 2026-02-25T15:05:40.353 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 11 0 2026-02-25T15:05:40.353 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 5 0 2026-02-25T15:05:40.353 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 8 0 2026-02-25T15:05:40.353 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 6 0 2026-02-25T15:05:40.353 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 3 0 2026-02-25T15:05:40.353 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 4 0 2026-02-25T15:05:40.353 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 12 0 2026-02-25T15:05:40.353 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 9 0 2026-02-25T15:05:40.353 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 2 0 2026-02-25T15:05:40.353 INFO:tasks.workunit.client.0.trial189.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2026-02-25T15:05:40.353 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (818 ms) 2026-02-25T15:05:40.353 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [----------] 3 tests from LibRadosListEC (2465 ms total) 2026-02-25T15:05:40.354 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 2026-02-25T15:05:40.354 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [----------] 1 test from LibRadosListNP 2026-02-25T15:05:40.354 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2026-02-25T15:05:40.354 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3144 ms) 2026-02-25T15:05:40.354 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [----------] 1 test from LibRadosListNP (3144 ms total) 2026-02-25T15:05:40.354 INFO:tasks.workunit.client.0.trial189.stdout: api_list: 2026-02-25T15:05:40.354 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [----------] Global test environment tear-down 2026-02-25T15:05:40.354 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [==========] 11 tests from 3 test suites ran. (209777 ms total) 2026-02-25T15:05:40.354 INFO:tasks.workunit.client.0.trial189.stdout: api_list: [ PASSED ] 11 tests. 2026-02-25T15:05:51.170 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:05:04.677 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:51.199 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:05:04.706 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:51.222 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:05:04.729 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:51.226 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:05:04.728 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:51.226 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:05:04.695 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:51.226 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:05:04.696 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:51.227 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:05:04.729 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:51.247 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:05:04.716 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:05:51.249 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:05:04.751 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:11.237 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:05:24.744 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:11.238 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:05:24.745 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:11.260 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:05:24.767 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:11.265 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:05:24.768 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:11.266 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:05:24.768 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:11.269 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:05:24.738 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:11.270 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:05:24.739 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:11.293 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:05:24.796 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:11.296 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:05:24.765 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:21.524 INFO:tasks.workunit.client.0.trial189.stdout:api_aio: [ OK ] LibRadosAio.CancelBeforeComplete (3057 ms) 2026-02-25T15:06:21.525 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAio.CancelAfterComplete 2026-02-25T15:06:21.525 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAio.CancelAfterComplete (2731 ms) 2026-02-25T15:06:21.525 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [----------] 29 tests from LibRadosAio (137552 ms total) 2026-02-25T15:06:21.525 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: 2026-02-25T15:06:21.525 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2026-02-25T15:06:21.525 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2026-02-25T15:06:21.525 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7175 ms) 2026-02-25T15:06:21.525 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2026-02-25T15:06:21.525 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (7117 ms) 2026-02-25T15:06:21.525 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2026-02-25T15:06:21.526 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (6716 ms) 2026-02-25T15:06:21.526 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2026-02-25T15:06:21.526 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (6982 ms) 2026-02-25T15:06:21.526 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2026-02-25T15:06:21.526 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7158 ms) 2026-02-25T15:06:21.526 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2026-02-25T15:06:21.526 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (6997 ms) 2026-02-25T15:06:21.526 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2026-02-25T15:06:21.526 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (6930 ms) 2026-02-25T15:06:21.526 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2026-02-25T15:06:21.526 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7386 ms) 2026-02-25T15:06:21.526 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2026-02-25T15:06:21.527 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6762 ms) 2026-02-25T15:06:21.527 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2026-02-25T15:06:21.527 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (6928 ms) 2026-02-25T15:06:21.527 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2026-02-25T15:06:21.527 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7210 ms) 2026-02-25T15:06:21.527 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2026-02-25T15:06:21.527 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (7047 ms) 2026-02-25T15:06:21.527 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2026-02-25T15:06:21.527 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (7876 ms) 2026-02-25T15:06:21.527 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2026-02-25T15:06:21.527 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (6857 ms) 2026-02-25T15:06:21.527 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2026-02-25T15:06:21.528 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7464 ms) 2026-02-25T15:06:21.528 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2026-02-25T15:06:21.528 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (6803 ms) 2026-02-25T15:06:21.528 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (113414 ms total) 2026-02-25T15:06:21.528 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: 2026-02-25T15:06:21.528 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [----------] Global test environment tear-down 2026-02-25T15:06:21.528 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [==========] 45 tests from 2 test suites ran. (250967 ms total) 2026-02-25T15:06:21.528 INFO:tasks.workunit.client.0.trial189.stdout: api_aio: [ PASSED ] 45 tests. 2026-02-25T15:06:31.275 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:05:44.782 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:31.276 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:05:44.784 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:31.299 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:05:44.806 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:31.304 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:05:44.806 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:31.305 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:05:44.807 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:31.307 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:05:44.776 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:31.308 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:05:44.777 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:31.328 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:05:44.831 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:31.331 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:05:44.801 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:51.315 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:06:04.822 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:51.316 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:06:04.823 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:51.334 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:06:04.841 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:51.345 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:06:04.814 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:51.346 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:06:04.848 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:51.346 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:06:04.815 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:51.347 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:06:04.849 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:51.364 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:06:04.833 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:51.365 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:06:04.868 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:06:56.337 INFO:tasks.workunit.client.0.trial189.stdout:io.RoundTripAppendPP (2821 ms) 2026-02-25T15:06:56.337 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2026-02-25T15:06:56.337 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3119 ms) 2026-02-25T15:06:56.337 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2026-02-25T15:06:56.337 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3219 ms) 2026-02-25T15:06:56.337 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2026-02-25T15:06:56.337 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3092 ms) 2026-02-25T15:06:56.337 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2026-02-25T15:06:56.338 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: sending pool EIO time: 2026-02-25T15:04:52.133890+0000 2026-02-25T15:06:56.338 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: /ceph/rpmbuild/BUILD/ceph-20.2.0-676-gc824a7fa/src/test/librados/aio_cxx.cc:2430: Skipped 2026-02-25T15:06:56.338 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: eio flag missed all ios that already completed 2026-02-25T15:06:56.338 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: 2026-02-25T15:06:56.338 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ SKIPPED ] LibRadosAio.PoolEIOFlag (7744 ms) 2026-02-25T15:06:56.338 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2026-02-25T15:06:56.338 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (2859 ms) 2026-02-25T15:06:56.338 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (165535 ms total) 2026-02-25T15:06:56.338 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: 2026-02-25T15:06:56.338 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2026-02-25T15:06:56.338 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2026-02-25T15:06:56.339 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3102 ms) 2026-02-25T15:06:56.339 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2026-02-25T15:06:56.339 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (8946 ms) 2026-02-25T15:06:56.339 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2026-02-25T15:06:56.339 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (15374 ms) 2026-02-25T15:06:56.339 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2026-02-25T15:06:56.339 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (2648 ms) 2026-02-25T15:06:56.339 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (30071 ms total) 2026-02-25T15:06:56.339 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: 2026-02-25T15:06:56.339 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2026-02-25T15:06:56.339 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2026-02-25T15:06:56.339 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3107 ms) 2026-02-25T15:06:56.340 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3107 ms total) 2026-02-25T15:06:56.340 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: 2026-02-25T15:06:56.340 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2026-02-25T15:06:56.340 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2026-02-25T15:06:56.340 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (14178 ms) 2026-02-25T15:06:56.340 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2026-02-25T15:06:56.340 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6943 ms) 2026-02-25T15:06:56.340 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2026-02-25T15:06:56.340 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (6916 ms) 2026-02-25T15:06:56.340 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2026-02-25T15:06:56.340 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7270 ms) 2026-02-25T15:06:56.340 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2026-02-25T15:06:56.341 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (2671 ms) 2026-02-25T15:06:56.341 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2026-02-25T15:06:56.341 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7463 ms) 2026-02-25T15:06:56.341 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2026-02-25T15:06:56.341 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6809 ms) 2026-02-25T15:06:56.341 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2026-02-25T15:06:56.341 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (6811 ms) 2026-02-25T15:06:56.341 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2026-02-25T15:06:56.341 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7140 ms) 2026-02-25T15:06:56.341 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2026-02-25T15:06:56.341 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (6972 ms) 2026-02-25T15:06:56.341 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2026-02-25T15:06:56.341 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (6958 ms) 2026-02-25T15:06:56.342 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2026-02-25T15:06:56.342 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (6928 ms) 2026-02-25T15:07:11.352 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:06:24.858 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:11.353 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:06:24.860 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:11.371 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:06:24.878 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:11.380 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:06:24.883 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:11.382 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:06:24.851 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:11.382 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:06:24.884 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:11.383 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:06:24.852 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:11.401 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:06:24.903 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:11.404 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:06:24.873 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:30.510 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:06:44.017 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:30.515 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:06:44.022 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:30.517 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:06:44.020 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:30.518 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:06:44.021 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:30.522 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:06:43.991 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:30.552 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:06:44.021 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:30.570 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:06:44.077 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:30.572 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:06:44.074 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:30.578 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:06:44.047 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:31.362 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:06:44.869 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:31.365 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:06:44.834 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:31.366 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:06:44.868 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:31.366 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:06:44.874 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:31.368 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:06:44.837 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:31.370 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:06:44.872 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:31.397 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:06:44.904 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:31.400 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:06:44.903 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:31.401 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:06:44.870 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:51.429 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:07:04.936 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:51.431 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:07:04.938 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:51.453 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:07:04.960 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:51.459 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:07:04.962 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:51.460 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:07:04.962 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:51.460 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:07:04.929 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:51.461 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:07:04.930 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:51.483 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:07:04.985 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:51.483 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:07:04.952 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:07:51.630 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2026-02-25T15:07:51.631 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7378 ms) 2026-02-25T15:07:51.631 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2026-02-25T15:07:51.631 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (2637 ms) 2026-02-25T15:07:51.631 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2026-02-25T15:07:51.631 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7320 ms) 2026-02-25T15:07:51.631 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2026-02-25T15:07:51.631 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (6838 ms) 2026-02-25T15:07:51.631 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2026-02-25T15:07:51.632 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (6894 ms) 2026-02-25T15:07:51.632 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2026-02-25T15:07:51.632 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7397 ms) 2026-02-25T15:07:51.632 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2026-02-25T15:07:51.632 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (6726 ms) 2026-02-25T15:07:51.632 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2026-02-25T15:07:51.632 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (6926 ms) 2026-02-25T15:07:51.632 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139185 ms total) 2026-02-25T15:07:51.632 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: 2026-02-25T15:07:51.632 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [----------] 6 tests from LibRadosAio/Cancel 2026-02-25T15:07:51.632 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio/Cancel.BeforeSubmit/aio_cancel 2026-02-25T15:07:51.632 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio/Cancel.BeforeSubmit/aio_cancel (0 ms) 2026-02-25T15:07:51.633 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio/Cancel.BeforeSubmit/cancel 2026-02-25T15:07:51.633 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio/Cancel.BeforeSubmit/cancel (0 ms) 2026-02-25T15:07:51.633 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio/Cancel.BeforeComplete/aio_cancel 2026-02-25T15:07:51.633 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio/Cancel.BeforeComplete/aio_cancel (0 ms) 2026-02-25T15:07:51.633 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio/Cancel.BeforeComplete/cancel 2026-02-25T15:07:51.633 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio/Cancel.BeforeComplete/cancel (0 ms) 2026-02-25T15:07:51.633 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio/Cancel.AfterComplete/aio_cancel 2026-02-25T15:07:51.633 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio/Cancel.AfterComplete/aio_cancel (67 ms) 2026-02-25T15:07:51.633 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ RUN ] LibRadosAio/Cancel.AfterComplete/cancel 2026-02-25T15:07:51.633 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ OK ] LibRadosAio/Cancel.AfterComplete/cancel (5 ms) 2026-02-25T15:07:51.633 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [----------] 6 tests from LibRadosAio/Cancel (73 ms total) 2026-02-25T15:07:51.633 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: 2026-02-25T15:07:51.634 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [----------] Global test environment tear-down 2026-02-25T15:07:51.634 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [==========] 63 tests from 5 test suites ran. (341071 ms total) 2026-02-25T15:07:51.634 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ PASSED ] 62 tests. 2026-02-25T15:07:51.634 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ SKIPPED ] 1 test, listed below: 2026-02-25T15:07:51.634 INFO:tasks.workunit.client.0.trial189.stdout: api_aio_pp: [ SKIPPED ] LibRadosAio.PoolEIOFlag 2026-02-25T15:07:51.638 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.638 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49178 2026-02-25T15:07:51.638 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49178 2026-02-25T15:07:51.638 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.638 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=50011 2026-02-25T15:07:51.638 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 50011 2026-02-25T15:07:51.639 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.639 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49351 2026-02-25T15:07:51.639 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49351 2026-02-25T15:07:51.639 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.639 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=50067 2026-02-25T15:07:51.639 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 50067 2026-02-25T15:07:51.639 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.639 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49448 2026-02-25T15:07:51.639 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49448 2026-02-25T15:07:51.639 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.639 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49853 2026-02-25T15:07:51.639 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49853 2026-02-25T15:07:51.640 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.640 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=50133 2026-02-25T15:07:51.640 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 50133 2026-02-25T15:07:51.640 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.640 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49384 2026-02-25T15:07:51.640 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49384 2026-02-25T15:07:51.640 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.640 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49727 2026-02-25T15:07:51.640 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49727 2026-02-25T15:07:51.640 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.640 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49979 2026-02-25T15:07:51.640 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49979 2026-02-25T15:07:51.640 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.641 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49819 2026-02-25T15:07:51.641 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49819 2026-02-25T15:07:51.641 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.641 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=50038 2026-02-25T15:07:51.641 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 50038 2026-02-25T15:07:51.641 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.641 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49663 2026-02-25T15:07:51.641 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49663 2026-02-25T15:07:51.641 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.641 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49312 2026-02-25T15:07:51.641 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49312 2026-02-25T15:07:51.641 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.641 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=50106 2026-02-25T15:07:51.642 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 50106 2026-02-25T15:07:51.642 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.642 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49577 2026-02-25T15:07:51.642 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49577 2026-02-25T15:07:51.642 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.642 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49150 2026-02-25T15:07:51.642 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49150 2026-02-25T15:07:51.642 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.642 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49209 2026-02-25T15:07:51.642 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49209 2026-02-25T15:07:51.642 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.642 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49500 2026-02-25T15:07:51.642 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49500 2026-02-25T15:07:51.643 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.643 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49251 2026-02-25T15:07:51.643 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49251 2026-02-25T15:07:51.643 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.643 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49606 2026-02-25T15:07:51.643 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49606 2026-02-25T15:07:51.643 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.643 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49628 2026-02-25T15:07:51.643 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49628 2026-02-25T15:07:51.643 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.643 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=49948 2026-02-25T15:07:51.643 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 49948 2026-02-25T15:07:51.643 INFO:tasks.workunit.client.0.trial189.stderr:+ for t in "${!pids[@]}" 2026-02-25T15:07:51.644 INFO:tasks.workunit.client.0.trial189.stderr:+ pid=50168 2026-02-25T15:07:51.644 INFO:tasks.workunit.client.0.trial189.stderr:+ wait 50168 2026-02-25T15:07:51.644 INFO:tasks.workunit.client.0.trial189.stderr:+ exit 0 2026-02-25T15:07:51.644 INFO:tasks.workunit.client.0.trial189.stderr:+ cleanup 2026-02-25T15:07:51.644 INFO:tasks.workunit.client.0.trial189.stderr:+ pkill -P 49144 2026-02-25T15:07:51.645 INFO:tasks.workunit.client.0.trial189.stderr:+ true 2026-02-25T15:07:51.645 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-25T15:07:51.646 DEBUG:teuthology.orchestra.run.trial189:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-25T15:07:51.672 INFO:tasks.workunit:Stopping ['rados/test.sh'] on client.0... 2026-02-25T15:07:51.673 DEBUG:teuthology.orchestra.run.trial189:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-25T15:07:52.052 DEBUG:teuthology.parallel:result is None 2026-02-25T15:07:52.053 DEBUG:teuthology.orchestra.run.trial189:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-25T15:07:52.079 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-25T15:07:52.079 DEBUG:teuthology.orchestra.run.trial189:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-25T15:07:52.135 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-25T15:07:52.136 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-25T15:07:52.148 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-25T15:07:52.148 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T15:07:52.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.220+0000 7f8392204640 10 monclient: get_monmap_and_config 2026-02-25T15:07:52.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.220+0000 7f8392204640 10 monclient: build_initial_monmap 2026-02-25T15:07:52.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.220+0000 7f8392204640 10 monclient: monmap: 2026-02-25T15:07:52.223 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:52.223 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.223 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:52.221842+0000 2026-02-25T15:07:52.223 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:52.221842+0000 2026-02-25T15:07:52.223 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:52.223 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.223 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:52.223 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:52.223 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:52.224 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.220+0000 7f8392204640 1 Processor -- start 2026-02-25T15:07:52.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.220+0000 7f8392204640 1 -- start start 2026-02-25T15:07:52.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.220+0000 7f8392204640 10 monclient: init 2026-02-25T15:07:52.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.221+0000 7f8392204640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:52.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.221+0000 7f8392204640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:07:52.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.221+0000 7f8392204640 10 monclient(hunting): picked mon.noname-c con 0x7f838c058da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:52.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.221+0000 7f8392204640 10 monclient(hunting): picked mon.noname-a con 0x7f838c0b0410 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:52.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.221+0000 7f8392204640 10 monclient(hunting): picked mon.noname-b con 0x7f838c1626a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:52.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.221+0000 7f8392204640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f838c1655c0 con 0x7f838c0b0410 2026-02-25T15:07:52.226 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.221+0000 7f8392204640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f838c0b12b0 con 0x7f838c1626a0 2026-02-25T15:07:52.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.221+0000 7f8392204640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f838c0b1590 con 0x7f838c058da0 2026-02-25T15:07:52.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.221+0000 7f8392204640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:52.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.221+0000 7f8392204640 10 monclient(hunting): authenticate will time out at 3361.822262s 2026-02-25T15:07:52.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.221+0000 7f838bfff640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f838c1626a0 0x7f838c1c6d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:34080/0 (socket says 10.20.193.90:34080) 2026-02-25T15:07:52.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.221+0000 7f838bfff640 1 -- 10.20.193.90:0/3490529845 learned_addr learned my addr 10.20.193.90:0/3490529845 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:52.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.221+0000 7f838affd640 1 --1- 10.20.193.90:0/3490529845 >> v1:10.20.193.90:6789/0 conn(0x7f838c0b0410 0x7f838c164eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:32918/0 (socket says 10.20.193.90:32918) 2026-02-25T15:07:52.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 1 -- 10.20.193.90:0/3490529845 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3110106765 0 0) 0x7f838c0b12b0 con 0x7f838c1626a0 2026-02-25T15:07:52.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 10 monclient(hunting): my global_id is 27391 2026-02-25T15:07:52.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 1 -- 10.20.193.90:0/3490529845 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8370003610 con 0x7f838c1626a0 2026-02-25T15:07:52.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 1 -- 10.20.193.90:0/3490529845 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1607184130 0 0) 0x7f838c1655c0 con 0x7f838c0b0410 2026-02-25T15:07:52.227 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 10 monclient(hunting): my global_id is 30759 2026-02-25T15:07:52.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 1 -- 10.20.193.90:0/3490529845 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f838c0b12b0 con 0x7f838c0b0410 2026-02-25T15:07:52.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 1 -- 10.20.193.90:0/3490529845 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1389841360 0 0) 0x7f838c0b1590 con 0x7f838c058da0 2026-02-25T15:07:52.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 10 monclient(hunting): my global_id is 28529 2026-02-25T15:07:52.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.223+0000 7f838a7fc640 1 -- 10.20.193.90:0/3490529845 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f838c1655c0 con 0x7f838c058da0 2026-02-25T15:07:52.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f838a7fc640 1 -- 10.20.193.90:0/3490529845 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 104043549 0 0) 0x7f8370003610 con 0x7f838c1626a0 2026-02-25T15:07:52.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f838a7fc640 1 -- 10.20.193.90:0/3490529845 >> v1:10.20.193.94:6789/0 conn(0x7f838c058da0 legacy=0x7f838c1647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f838a7fc640 1 -- 10.20.193.90:0/3490529845 >> v1:10.20.193.90:6789/0 conn(0x7f838c0b0410 legacy=0x7f838c164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f838a7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:07:52.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f838a7fc640 1 monclient: found mon.noname-b 2026-02-25T15:07:52.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f838a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:52.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f838a7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:07:52.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f838a7fc640 1 -- 10.20.193.90:0/3490529845 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838c0b1870 con 0x7f838c1626a0 2026-02-25T15:07:52.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f838a7fc640 10 monclient: _finish_auth 0 2026-02-25T15:07:52.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f838a7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:07:52.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f838a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:52.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f838a7fc640 1 -- 10.20.193.90:0/3490529845 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8380003170 con 0x7f838c1626a0 2026-02-25T15:07:52.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f838a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:52.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f838a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:52.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f838a7fc640 10 monclient: dump: 2026-02-25T15:07:52.233 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.233 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.233 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.233 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.233 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.233 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.233 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.233 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.233 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.233 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f8392204640 5 monclient: authenticate success, global_id 27391 2026-02-25T15:07:52.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.225+0000 7f8392204640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:52.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.226+0000 7f838a7fc640 1 -- 10.20.193.90:0/3490529845 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f83800033c0 con 0x7f838c1626a0 2026-02-25T15:07:52.233 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.226+0000 7f838a7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:52.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.226+0000 7f838a7fc640 1 -- 10.20.193.90:0/3490529845 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8380003800 con 0x7f838c1626a0 2026-02-25T15:07:52.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.226+0000 7f838a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:52.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.226+0000 7f838a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:52.234 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.226+0000 7f838a7fc640 10 monclient: dump: 2026-02-25T15:07:52.234 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.234 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.234 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.234 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.234 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.234 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.234 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.234 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.234 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.234 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.226+0000 7f8392204640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:52.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.226+0000 7f8392204640 4 set_mon_vals no callback set 2026-02-25T15:07:52.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.226+0000 7f8392204640 10 monclient: shutdown 2026-02-25T15:07:52.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.226+0000 7f8392204640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:52.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.226+0000 7f8392204640 1 -- 10.20.193.90:0/3490529845 >> v1:10.20.193.93:6789/0 conn(0x7f838c1626a0 legacy=0x7f838c1c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.226+0000 7f8392204640 1 -- 10.20.193.90:0/3490529845 shutdown_connections 2026-02-25T15:07:52.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.226+0000 7f8392204640 1 -- 10.20.193.90:0/3490529845 >> 10.20.193.90:0/3490529845 conn(0x7f838c0b65f0 msgr2=0x7f838c0b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:52.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.226+0000 7f8392204640 1 -- 10.20.193.90:0/3490529845 shutdown_connections 2026-02-25T15:07:52.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.226+0000 7f8392204640 1 -- 10.20.193.90:0/3490529845 wait complete. 2026-02-25T15:07:52.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.227+0000 7f8392204640 10 monclient: build_initial_monmap 2026-02-25T15:07:52.235 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.227+0000 7f8392204640 10 monclient: monmap: 2026-02-25T15:07:52.235 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:52.235 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:52.235 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:52.236 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:52.236 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:52.236 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:52.236 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:52.236 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:52.236 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:52.236 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.227+0000 7f8392204640 1 librados: starting msgr at 2026-02-25T15:07:52.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.227+0000 7f8392204640 1 librados: starting objecter 2026-02-25T15:07:52.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.227+0000 7f8392204640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:52.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.227+0000 7f8392204640 1 Processor -- start 2026-02-25T15:07:52.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.227+0000 7f8392204640 1 -- start start 2026-02-25T15:07:52.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.227+0000 7f8392204640 1 librados: setting wanted keys 2026-02-25T15:07:52.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.227+0000 7f8392204640 1 librados: calling monclient init 2026-02-25T15:07:52.236 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.227+0000 7f8392204640 10 monclient: init 2026-02-25T15:07:52.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8392204640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:52.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8392204640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:07:52.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8392204640 10 monclient(hunting): picked mon.noname-a con 0x7f838c058da0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:52.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8392204640 10 monclient(hunting): picked mon.noname-c con 0x7f838c0b0410 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:52.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8392204640 10 monclient(hunting): picked mon.noname-b con 0x7f838c1626a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:52.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8392204640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f838c131260 con 0x7f838c058da0 2026-02-25T15:07:52.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8392204640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f838c0aff60 con 0x7f838c1626a0 2026-02-25T15:07:52.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8392204640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f838c13b7f0 con 0x7f838c0b0410 2026-02-25T15:07:52.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8392204640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:52.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8392204640 10 monclient(hunting): authenticate will time out at 3361.829358s 2026-02-25T15:07:52.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f838b7fe640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f838c058da0 0x7f838c13b0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:32928/0 (socket says 10.20.193.90:32928) 2026-02-25T15:07:52.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f838b7fe640 1 -- 10.20.193.90:0/1827392292 learned_addr learned my addr 10.20.193.90:0/1827392292 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:52.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8367fff640 10 client.?.objecter ms_handle_connect 0x7f838c058da0 2026-02-25T15:07:52.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8367fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:52.237 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8367fff640 10 client.?.objecter ms_handle_connect 0x7f838c1626a0 2026-02-25T15:07:52.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8367fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:52.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8367fff640 10 client.?.objecter ms_handle_connect 0x7f838c0b0410 2026-02-25T15:07:52.238 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.228+0000 7f8367fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:52.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.229+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 774333796 0 0) 0x7f838c13b7f0 con 0x7f838c0b0410 2026-02-25T15:07:52.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.229+0000 7f8367fff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.229+0000 7f8367fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.229+0000 7f8367fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.229+0000 7f8367fff640 10 monclient(hunting): my global_id is 28532 2026-02-25T15:07:52.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.229+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f835c003610 con 0x7f838c0b0410 2026-02-25T15:07:52.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.229+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 501513294 0 0) 0x7f838c131260 con 0x7f838c058da0 2026-02-25T15:07:52.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.229+0000 7f8367fff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.229+0000 7f8367fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.229+0000 7f8367fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.229+0000 7f8367fff640 10 monclient(hunting): my global_id is 30762 2026-02-25T15:07:52.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.229+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f838c13b7f0 con 0x7f838c058da0 2026-02-25T15:07:52.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.230+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2375669227 0 0) 0x7f838c0aff60 con 0x7f838c1626a0 2026-02-25T15:07:52.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.230+0000 7f8367fff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.230+0000 7f8367fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.230+0000 7f8367fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.241 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.230+0000 7f8367fff640 10 monclient(hunting): my global_id is 27394 2026-02-25T15:07:52.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.230+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f838c131260 con 0x7f838c1626a0 2026-02-25T15:07:52.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4283934477 0 0) 0x7f838c13b7f0 con 0x7f838c058da0 2026-02-25T15:07:52.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f838c0aff60 con 0x7f838c058da0 2026-02-25T15:07:52.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8378003170 con 0x7f838c058da0 2026-02-25T15:07:52.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:52.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:52.242 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 10 monclient(hunting): dump: 2026-02-25T15:07:52.242 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.242 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.242 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.242 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.242 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.242 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.243 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.243 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.243 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.243 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:52.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1740167743 0 0) 0x7f835c003610 con 0x7f838c0b0410 2026-02-25T15:07:52.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f838c13b7f0 con 0x7f838c0b0410 2026-02-25T15:07:52.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f837c0030b0 con 0x7f838c0b0410 2026-02-25T15:07:52.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:52.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:52.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 10 monclient(hunting): dump: 2026-02-25T15:07:52.243 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.243 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.243 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.244 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.244 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.244 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.244 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.244 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.244 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.244 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 589939855 0 0) 0x7f838c131260 con 0x7f838c1626a0 2026-02-25T15:07:52.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f835c003610 con 0x7f838c1626a0 2026-02-25T15:07:52.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8380003570 con 0x7f838c1626a0 2026-02-25T15:07:52.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:52.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:52.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.232+0000 7f8367fff640 10 monclient(hunting): dump: 2026-02-25T15:07:52.244 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.244 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.245 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.245 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.245 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.245 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.245 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.245 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.245 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.245 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2772404954 0 0) 0x7f838c13b7f0 con 0x7f838c0b0410 2026-02-25T15:07:52.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 >> v1:10.20.193.93:6789/0 conn(0x7f838c1626a0 legacy=0x7f838c14aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 >> v1:10.20.193.90:6789/0 conn(0x7f838c058da0 legacy=0x7f838c13b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8367fff640 10 monclient: _finish_hunting 0 2026-02-25T15:07:52.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8367fff640 1 monclient: found mon.c 2026-02-25T15:07:52.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8367fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:52.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8367fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:52.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838c147d30 con 0x7f838c0b0410 2026-02-25T15:07:52.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8367fff640 10 monclient: _finish_auth 0 2026-02-25T15:07:52.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8367fff640 10 monclient: _check_auth_tickets 2026-02-25T15:07:52.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8367fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:52.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8392204640 5 monclient: authenticate success, global_id 28532 2026-02-25T15:07:52.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8392204640 10 monclient: _renew_subs 2026-02-25T15:07:52.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8392204640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:52.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8392204640 1 -- 10.20.193.90:0/1827392292 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f838c148010 con 0x7f838c0b0410 2026-02-25T15:07:52.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8392204640 10 client.28532.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:52.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8392204640 10 monclient: _renew_subs 2026-02-25T15:07:52.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8392204640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:52.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8392204640 1 -- 10.20.193.90:0/1827392292 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f838c1485d0 con 0x7f838c0b0410 2026-02-25T15:07:52.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.234+0000 7f8392204640 1 librados: init done 2026-02-25T15:07:52.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.235+0000 7f8392204640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:52.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.235+0000 7f8392204640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:52.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.235+0000 7f8392204640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:52.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.235+0000 7f8392204640 1 -- 10.20.193.90:0/1827392292 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f838c0a9490 con 0x7f838c0b0410 2026-02-25T15:07:52.333 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.331+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f837c0048d0 con 0x7f838c0b0410 2026-02-25T15:07:52.333 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.331+0000 7f8367fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:52.333 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.331+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f837c004c20 con 0x7f838c0b0410 2026-02-25T15:07:52.333 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.331+0000 7f8367fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:52.333 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.331+0000 7f8367fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:52.334 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.331+0000 7f8367fff640 10 monclient: dump: 2026-02-25T15:07:52.334 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.334 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.334 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.334 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.334 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.334 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.334 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.334 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.334 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.334 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.334 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.331+0000 7f8389ffb640 4 set_mon_vals no callback set 2026-02-25T15:07:52.335 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.331+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f837c01e100 con 0x7f838c0b0410 2026-02-25T15:07:52.336 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.331+0000 7f838b7fe640 10 monclient: get_auth_request con 0x7f835c07a110 auth_method 0 2026-02-25T15:07:52.336 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.332+0000 7f8367fff640 10 client.28532.objecter ms_handle_connect 0x7f835c07a110 2026-02-25T15:07:52.336 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.332+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(337..337 src has 1..337) ==== 8959+0+0 (unknown 3556021173 0 0) 0x7f837c095920 con 0x7f838c0b0410 2026-02-25T15:07:52.336 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.332+0000 7f8367fff640 10 client.28532.objecter ms_dispatch2 0x7f838c000b90 osd_map(337..337 src has 1..337) 2026-02-25T15:07:52.336 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.332+0000 7f8367fff640 3 client.28532.objecter handle_osd_map got epochs [337,337] > 0 2026-02-25T15:07:52.336 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.332+0000 7f8367fff640 3 client.28532.objecter handle_osd_map decoding full epoch 337 2026-02-25T15:07:52.336 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.332+0000 7f8367fff640 20 client.28532.objecter dump_active .. 0 homeless 2026-02-25T15:07:52.337 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.334+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f838c0a9490 con 0x7f838c0b0410 2026-02-25T15:07:52.337 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.334+0000 7f8367fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:52.337 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.334+0000 7f8367fff640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:52.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.437+0000 7f8392204640 1 -- 10.20.193.90:0/1827392292 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f838c148810 con 0x7f835c07a110 2026-02-25T15:07:52.442 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.440+0000 7f8367fff640 1 -- 10.20.193.90:0/1827392292 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+125395 (unknown 2965378022 0 1250104598) 0x7f838c148810 con 0x7f835c07a110 2026-02-25T15:07:52.443 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:07:52.444 INFO:teuthology.orchestra.run.trial090.stderr:dumped all 2026-02-25T15:07:52.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.440+0000 7f8392204640 10 librados: watch_flush enter 2026-02-25T15:07:52.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.440+0000 7f8392204640 10 librados: watch_flush exit 2026-02-25T15:07:52.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.440+0000 7f8392204640 20 client.28532.objecter shutdown clearing up homeless session... 2026-02-25T15:07:52.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.440+0000 7f8392204640 10 client.28532.objecter successfully canceled tick 2026-02-25T15:07:52.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.440+0000 7f8392204640 1 -- 10.20.193.90:0/1827392292 >> v1:10.20.193.189:6800/3647375982 conn(0x7f835c07a110 legacy=0x7f835c09a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.440+0000 7f8392204640 10 monclient: shutdown 2026-02-25T15:07:52.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.440+0000 7f8392204640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:52.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.440+0000 7f8392204640 1 -- 10.20.193.90:0/1827392292 >> v1:10.20.193.94:6789/0 conn(0x7f838c0b0410 legacy=0x7f838c131740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.440+0000 7f8392204640 1 -- 10.20.193.90:0/1827392292 shutdown_connections 2026-02-25T15:07:52.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.440+0000 7f8392204640 1 -- 10.20.193.90:0/1827392292 >> 10.20.193.90:0/1827392292 conn(0x7f838c0b65f0 msgr2=0x7f838c055980 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:52.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.440+0000 7f8392204640 1 -- 10.20.193.90:0/1827392292 shutdown_connections 2026-02-25T15:07:52.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.440+0000 7f8392204640 1 -- 10.20.193.90:0/1827392292 wait complete. 2026-02-25T15:07:52.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.440+0000 7f8392204640 1 librados: shutdown 2026-02-25T15:07:52.451 INFO:teuthology.orchestra.run.trial090.stdout:{"pg_ready":true,"pg_map":{"version":530,"stamp":"2026-02-25T15:07:51.593123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":56,"num_read_kb":47,"num_write":70,"num_write_kb":597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":557000,"kb_used_data":107592,"kb_used_omap":21829,"kb_used_meta":427322,"kb_avail":1509392440,"statfs":{"total":1546188226560,"available":1545617858560,"internally_reserved":0,"allocated":110174208,"data_stored":68392257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22353345,"internal_metadata":437578303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":78,"apply_latency_ms":78,"commit_latency_ns":78000000,"apply_latency_ns":78000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.158859"},"pg_stats":[{"pgid":"25.5","version":"0'0","reported_seq":619,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.437533+0000","last_change":"2026-02-25T15:02:14.833079+0000","last_active":"2026-02-25T15:07:48.437533+0000","last_peered":"2026-02-25T15:07:48.437533+0000","last_clean":"2026-02-25T15:07:48.437533+0000","last_became_active":"2026-02-25T15:02:14.830708+0000","last_became_peered":"2026-02-25T15:02:14.830708+0000","last_unstale":"2026-02-25T15:07:48.437533+0000","last_undegraded":"2026-02-25T15:07:48.437533+0000","last_fullsized":"2026-02-25T15:07:48.437533+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:21:07.436552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,8],"acting":[14,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.569659+0000","last_change":"2026-02-25T15:02:14.703677+0000","last_active":"2026-02-25T15:07:50.569659+0000","last_peered":"2026-02-25T15:07:50.569659+0000","last_clean":"2026-02-25T15:07:50.569659+0000","last_became_active":"2026-02-25T15:02:14.686130+0000","last_became_peered":"2026-02-25T15:02:14.686130+0000","last_unstale":"2026-02-25T15:07:50.569659+0000","last_undegraded":"2026-02-25T15:07:50.569659+0000","last_fullsized":"2026-02-25T15:07:50.569659+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:37:58.016312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7],"acting":[9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.444074+0000","last_change":"2026-02-25T15:02:14.184346+0000","last_active":"2026-02-25T15:07:48.444074+0000","last_peered":"2026-02-25T15:07:48.444074+0000","last_clean":"2026-02-25T15:07:48.444074+0000","last_became_active":"2026-02-25T15:02:14.182253+0000","last_became_peered":"2026-02-25T15:02:14.182253+0000","last_unstale":"2026-02-25T15:07:48.444074+0000","last_undegraded":"2026-02-25T15:07:48.444074+0000","last_fullsized":"2026-02-25T15:07:48.444074+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:10:33.826830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.442617+0000","last_change":"2026-02-25T15:02:14.262914+0000","last_active":"2026-02-25T15:07:48.442617+0000","last_peered":"2026-02-25T15:07:48.442617+0000","last_clean":"2026-02-25T15:07:48.442617+0000","last_became_active":"2026-02-25T15:02:14.258456+0000","last_became_peered":"2026-02-25T15:02:14.258456+0000","last_unstale":"2026-02-25T15:07:48.442617+0000","last_undegraded":"2026-02-25T15:07:48.442617+0000","last_fullsized":"2026-02-25T15:07:48.442617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:29:58.464087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"33'11","reported_seq":645,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.505217+0000","last_change":"2026-02-25T15:02:15.833921+0000","last_active":"2026-02-25T15:07:49.505217+0000","last_peered":"2026-02-25T15:07:49.505217+0000","last_clean":"2026-02-25T15:07:49.505217+0000","last_became_active":"2026-02-25T15:02:15.819620+0000","last_became_peered":"2026-02-25T15:02:15.819620+0000","last_unstale":"2026-02-25T15:07:49.505217+0000","last_undegraded":"2026-02-25T15:07:49.505217+0000","last_fullsized":"2026-02-25T15:07:49.505217+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:40:10.738663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,13],"acting":[10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":616,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.562854+0000","last_change":"2026-02-25T15:02:13.775077+0000","last_active":"2026-02-25T15:07:49.562854+0000","last_peered":"2026-02-25T15:07:49.562854+0000","last_clean":"2026-02-25T15:07:49.562854+0000","last_became_active":"2026-02-25T15:02:13.773022+0000","last_became_peered":"2026-02-25T15:02:13.773022+0000","last_unstale":"2026-02-25T15:07:49.562854+0000","last_undegraded":"2026-02-25T15:07:49.562854+0000","last_fullsized":"2026-02-25T15:07:49.562854+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:04:02.164409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":620,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.558382+0000","last_change":"2026-02-25T15:02:14.036683+0000","last_active":"2026-02-25T15:07:49.558382+0000","last_peered":"2026-02-25T15:07:49.558382+0000","last_clean":"2026-02-25T15:07:49.558382+0000","last_became_active":"2026-02-25T15:02:14.028952+0000","last_became_peered":"2026-02-25T15:02:14.028952+0000","last_unstale":"2026-02-25T15:07:49.558382+0000","last_undegraded":"2026-02-25T15:07:49.558382+0000","last_fullsized":"2026-02-25T15:07:49.558382+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:24:37.223453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,13],"acting":[5,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":619,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.584196+0000","last_change":"2026-02-25T15:02:14.938617+0000","last_active":"2026-02-25T15:07:50.584196+0000","last_peered":"2026-02-25T15:07:50.584196+0000","last_clean":"2026-02-25T15:07:50.584196+0000","last_became_active":"2026-02-25T15:02:14.936242+0000","last_became_peered":"2026-02-25T15:02:14.936242+0000","last_unstale":"2026-02-25T15:07:50.584196+0000","last_undegraded":"2026-02-25T15:07:50.584196+0000","last_fullsized":"2026-02-25T15:07:50.584196+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:59:06.072509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.579130+0000","last_change":"2026-02-25T15:02:13.982149+0000","last_active":"2026-02-25T15:07:48.579130+0000","last_peered":"2026-02-25T15:07:48.579130+0000","last_clean":"2026-02-25T15:07:48.579130+0000","last_became_active":"2026-02-25T15:02:13.976916+0000","last_became_peered":"2026-02-25T15:02:13.976916+0000","last_unstale":"2026-02-25T15:07:48.579130+0000","last_undegraded":"2026-02-25T15:07:48.579130+0000","last_fullsized":"2026-02-25T15:07:48.579130+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:19:44.345322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":616,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.565247+0000","last_change":"2026-02-25T15:02:13.384676+0000","last_active":"2026-02-25T15:07:49.565247+0000","last_peered":"2026-02-25T15:07:49.565247+0000","last_clean":"2026-02-25T15:07:49.565247+0000","last_became_active":"2026-02-25T15:02:13.382450+0000","last_became_peered":"2026-02-25T15:02:13.382450+0000","last_unstale":"2026-02-25T15:07:49.565247+0000","last_undegraded":"2026-02-25T15:07:49.565247+0000","last_fullsized":"2026-02-25T15:07:49.565247+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:23:31.158323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10],"acting":[0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":617,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.605481+0000","last_change":"2026-02-25T15:02:13.940723+0000","last_active":"2026-02-25T15:07:49.605481+0000","last_peered":"2026-02-25T15:07:49.605481+0000","last_clean":"2026-02-25T15:07:49.605481+0000","last_became_active":"2026-02-25T15:02:13.934323+0000","last_became_peered":"2026-02-25T15:02:13.934323+0000","last_unstale":"2026-02-25T15:07:49.605481+0000","last_undegraded":"2026-02-25T15:07:49.605481+0000","last_fullsized":"2026-02-25T15:07:49.605481+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:14:02.898606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10],"acting":[11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":620,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.567595+0000","last_change":"2026-02-25T15:02:14.071335+0000","last_active":"2026-02-25T15:07:49.567595+0000","last_peered":"2026-02-25T15:07:49.567595+0000","last_clean":"2026-02-25T15:07:49.567595+0000","last_became_active":"2026-02-25T15:02:14.063362+0000","last_became_peered":"2026-02-25T15:02:14.063362+0000","last_unstale":"2026-02-25T15:07:49.567595+0000","last_undegraded":"2026-02-25T15:07:49.567595+0000","last_fullsized":"2026-02-25T15:07:49.567595+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:41:34.565838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,14],"acting":[5,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.440309+0000","last_change":"2026-02-25T15:02:13.851333+0000","last_active":"2026-02-25T15:07:48.440309+0000","last_peered":"2026-02-25T15:07:48.440309+0000","last_clean":"2026-02-25T15:07:48.440309+0000","last_became_active":"2026-02-25T15:02:13.849038+0000","last_became_peered":"2026-02-25T15:02:13.849038+0000","last_unstale":"2026-02-25T15:07:48.440309+0000","last_undegraded":"2026-02-25T15:07:48.440309+0000","last_fullsized":"2026-02-25T15:07:48.440309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:04:11.634095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":632,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.604755+0000","last_change":"2026-02-25T15:01:40.617020+0000","last_active":"2026-02-25T15:07:49.604755+0000","last_peered":"2026-02-25T15:07:49.604755+0000","last_clean":"2026-02-25T15:07:49.604755+0000","last_became_active":"2026-02-25T15:01:38.997857+0000","last_became_peered":"2026-02-25T15:01:38.997857+0000","last_unstale":"2026-02-25T15:07:49.604755+0000","last_undegraded":"2026-02-25T15:07:49.604755+0000","last_fullsized":"2026-02-25T15:07:49.604755+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:49:56.506302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062975700999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14],"acting":[11,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":617,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.546518+0000","last_change":"2026-02-25T15:02:13.715917+0000","last_active":"2026-02-25T15:07:49.546518+0000","last_peered":"2026-02-25T15:07:49.546518+0000","last_clean":"2026-02-25T15:07:49.546518+0000","last_became_active":"2026-02-25T15:02:13.713599+0000","last_became_peered":"2026-02-25T15:02:13.713599+0000","last_unstale":"2026-02-25T15:07:49.546518+0000","last_undegraded":"2026-02-25T15:07:49.546518+0000","last_fullsized":"2026-02-25T15:07:49.546518+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:15:39.798144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":636,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.567836+0000","last_change":"2026-02-25T15:01:41.128386+0000","last_active":"2026-02-25T15:07:50.567836+0000","last_peered":"2026-02-25T15:07:50.567836+0000","last_clean":"2026-02-25T15:07:50.567836+0000","last_became_active":"2026-02-25T15:01:38.782341+0000","last_became_peered":"2026-02-25T15:01:38.782341+0000","last_unstale":"2026-02-25T15:07:50.567836+0000","last_undegraded":"2026-02-25T15:07:50.567836+0000","last_fullsized":"2026-02-25T15:07:50.567836+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:07:32.765627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031008292,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,3],"acting":[12,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":619,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.563927+0000","last_change":"2026-02-25T15:02:13.858401+0000","last_active":"2026-02-25T15:07:50.563927+0000","last_peered":"2026-02-25T15:07:50.563927+0000","last_clean":"2026-02-25T15:07:50.563927+0000","last_became_active":"2026-02-25T15:02:13.846214+0000","last_became_peered":"2026-02-25T15:02:13.846214+0000","last_unstale":"2026-02-25T15:07:50.563927+0000","last_undegraded":"2026-02-25T15:07:50.563927+0000","last_fullsized":"2026-02-25T15:07:50.563927+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:30:33.308451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11],"acting":[3,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":634,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.596549+0000","last_change":"2026-02-25T15:01:41.143783+0000","last_active":"2026-02-25T15:07:50.596549+0000","last_peered":"2026-02-25T15:07:50.596549+0000","last_clean":"2026-02-25T15:07:50.596549+0000","last_became_active":"2026-02-25T15:01:38.731569+0000","last_became_peered":"2026-02-25T15:01:38.731569+0000","last_unstale":"2026-02-25T15:07:50.596549+0000","last_undegraded":"2026-02-25T15:07:50.596549+0000","last_fullsized":"2026-02-25T15:07:50.596549+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:00:25.099174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037093318,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15],"acting":[7,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":619,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.441335+0000","last_change":"2026-02-25T15:02:14.257499+0000","last_active":"2026-02-25T15:07:48.441335+0000","last_peered":"2026-02-25T15:07:48.441335+0000","last_clean":"2026-02-25T15:07:48.441335+0000","last_became_active":"2026-02-25T15:02:14.254843+0000","last_became_peered":"2026-02-25T15:02:14.254843+0000","last_unstale":"2026-02-25T15:07:48.441335+0000","last_undegraded":"2026-02-25T15:07:48.441335+0000","last_fullsized":"2026-02-25T15:07:48.441335+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:07:11.424289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2],"acting":[14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":634,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.593113+0000","last_change":"2026-02-25T15:01:41.139794+0000","last_active":"2026-02-25T15:07:50.593113+0000","last_peered":"2026-02-25T15:07:50.593113+0000","last_clean":"2026-02-25T15:07:50.593113+0000","last_became_active":"2026-02-25T15:01:38.780556+0000","last_became_peered":"2026-02-25T15:01:38.780556+0000","last_unstale":"2026-02-25T15:07:50.593113+0000","last_undegraded":"2026-02-25T15:07:50.593113+0000","last_fullsized":"2026-02-25T15:07:50.593113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:57:49.842137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041710525999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":616,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.560799+0000","last_change":"2026-02-25T15:02:14.919090+0000","last_active":"2026-02-25T15:07:49.560799+0000","last_peered":"2026-02-25T15:07:49.560799+0000","last_clean":"2026-02-25T15:07:49.560799+0000","last_became_active":"2026-02-25T15:02:14.894984+0000","last_became_peered":"2026-02-25T15:02:14.894984+0000","last_unstale":"2026-02-25T15:07:49.560799+0000","last_undegraded":"2026-02-25T15:07:49.560799+0000","last_fullsized":"2026-02-25T15:07:49.560799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:10:40.021124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":634,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.596471+0000","last_change":"2026-02-25T15:01:40.608781+0000","last_active":"2026-02-25T15:07:49.596471+0000","last_peered":"2026-02-25T15:07:49.596471+0000","last_clean":"2026-02-25T15:07:49.596471+0000","last_became_active":"2026-02-25T15:01:38.743941+0000","last_became_peered":"2026-02-25T15:01:38.743941+0000","last_unstale":"2026-02-25T15:07:49.596471+0000","last_undegraded":"2026-02-25T15:07:49.596471+0000","last_fullsized":"2026-02-25T15:07:49.596471+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:53:32.920667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.061484652000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13],"acting":[11,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":620,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.568600+0000","last_change":"2026-02-25T15:02:14.369913+0000","last_active":"2026-02-25T15:07:49.568600+0000","last_peered":"2026-02-25T15:07:49.568600+0000","last_clean":"2026-02-25T15:07:49.568600+0000","last_became_active":"2026-02-25T15:02:14.364385+0000","last_became_peered":"2026-02-25T15:02:14.364385+0000","last_unstale":"2026-02-25T15:07:49.568600+0000","last_undegraded":"2026-02-25T15:07:49.568600+0000","last_fullsized":"2026-02-25T15:07:49.568600+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:35:03.754835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":632,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.543792+0000","last_change":"2026-02-25T15:01:41.129594+0000","last_active":"2026-02-25T15:07:49.543792+0000","last_peered":"2026-02-25T15:07:49.543792+0000","last_clean":"2026-02-25T15:07:49.543792+0000","last_became_active":"2026-02-25T15:01:38.861789+0000","last_became_peered":"2026-02-25T15:01:38.861789+0000","last_unstale":"2026-02-25T15:07:49.543792+0000","last_undegraded":"2026-02-25T15:07:49.543792+0000","last_fullsized":"2026-02-25T15:07:49.543792+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:36:46.424677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032448785000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":617,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.598089+0000","last_change":"2026-02-25T15:02:15.000767+0000","last_active":"2026-02-25T15:07:49.598089+0000","last_peered":"2026-02-25T15:07:49.598089+0000","last_clean":"2026-02-25T15:07:49.598089+0000","last_became_active":"2026-02-25T15:02:14.998340+0000","last_became_peered":"2026-02-25T15:02:14.998340+0000","last_unstale":"2026-02-25T15:07:49.598089+0000","last_undegraded":"2026-02-25T15:07:49.598089+0000","last_fullsized":"2026-02-25T15:07:49.598089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:06:28.044352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5],"acting":[11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":632,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.562031+0000","last_change":"2026-02-25T15:01:41.116971+0000","last_active":"2026-02-25T15:07:49.562031+0000","last_peered":"2026-02-25T15:07:49.562031+0000","last_clean":"2026-02-25T15:07:49.562031+0000","last_became_active":"2026-02-25T15:01:38.737942+0000","last_became_peered":"2026-02-25T15:01:38.737942+0000","last_unstale":"2026-02-25T15:07:49.562031+0000","last_undegraded":"2026-02-25T15:07:49.562031+0000","last_fullsized":"2026-02-25T15:07:49.562031+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:51:33.138764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031295004000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13],"acting":[0,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":619,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.570012+0000","last_change":"2026-02-25T15:02:13.590713+0000","last_active":"2026-02-25T15:07:50.570012+0000","last_peered":"2026-02-25T15:07:50.570012+0000","last_clean":"2026-02-25T15:07:50.570012+0000","last_became_active":"2026-02-25T15:02:13.588299+0000","last_became_peered":"2026-02-25T15:02:13.588299+0000","last_unstale":"2026-02-25T15:07:50.570012+0000","last_undegraded":"2026-02-25T15:07:50.570012+0000","last_fullsized":"2026-02-25T15:07:50.570012+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:10:21.612478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":633,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.598833+0000","last_change":"2026-02-25T15:01:40.614625+0000","last_active":"2026-02-25T15:07:49.598833+0000","last_peered":"2026-02-25T15:07:49.598833+0000","last_clean":"2026-02-25T15:07:49.598833+0000","last_became_active":"2026-02-25T15:01:38.791923+0000","last_became_peered":"2026-02-25T15:01:38.791923+0000","last_unstale":"2026-02-25T15:07:49.598833+0000","last_undegraded":"2026-02-25T15:07:49.598833+0000","last_fullsized":"2026-02-25T15:07:49.598833+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:51:10.001377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062725172999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5],"acting":[11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"1.0","version":"24'32","reported_seq":680,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.555620+0000","last_change":"2026-02-25T15:01:30.868615+0000","last_active":"2026-02-25T15:07:49.555620+0000","last_peered":"2026-02-25T15:07:49.555620+0000","last_clean":"2026-02-25T15:07:49.555620+0000","last_became_active":"2026-02-25T15:01:30.851092+0000","last_became_peered":"2026-02-25T15:01:30.851092+0000","last_unstale":"2026-02-25T15:07:49.555620+0000","last_undegraded":"2026-02-25T15:07:49.555620+0000","last_fullsized":"2026-02-25T15:07:49.555620+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:29.405841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:29.405841+0000","last_clean_scrub_stamp":"2026-02-25T15:01:29.405841+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T02:31:46.315230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3],"acting":[8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.441645+0000","last_change":"2026-02-25T15:02:15.278232+0000","last_active":"2026-02-25T15:07:48.441645+0000","last_peered":"2026-02-25T15:07:48.441645+0000","last_clean":"2026-02-25T15:07:48.441645+0000","last_became_active":"2026-02-25T15:02:15.276113+0000","last_became_peered":"2026-02-25T15:02:15.276113+0000","last_unstale":"2026-02-25T15:07:48.441645+0000","last_undegraded":"2026-02-25T15:07:48.441645+0000","last_fullsized":"2026-02-25T15:07:48.441645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:15:46.080630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":619,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.434205+0000","last_change":"2026-02-25T15:02:14.248177+0000","last_active":"2026-02-25T15:07:48.434205+0000","last_peered":"2026-02-25T15:07:48.434205+0000","last_clean":"2026-02-25T15:07:48.434205+0000","last_became_active":"2026-02-25T15:02:14.243187+0000","last_became_peered":"2026-02-25T15:02:14.243187+0000","last_unstale":"2026-02-25T15:07:48.434205+0000","last_undegraded":"2026-02-25T15:07:48.434205+0000","last_fullsized":"2026-02-25T15:07:48.434205+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:34:38.222587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,10],"acting":[14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":619,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.436183+0000","last_change":"2026-02-25T15:02:13.817879+0000","last_active":"2026-02-25T15:07:48.436183+0000","last_peered":"2026-02-25T15:07:48.436183+0000","last_clean":"2026-02-25T15:07:48.436183+0000","last_became_active":"2026-02-25T15:02:13.815222+0000","last_became_peered":"2026-02-25T15:02:13.815222+0000","last_unstale":"2026-02-25T15:07:48.436183+0000","last_undegraded":"2026-02-25T15:07:48.436183+0000","last_fullsized":"2026-02-25T15:07:48.436183+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:31:52.870792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,9],"acting":[14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.567169+0000","last_change":"2026-02-25T15:02:15.251677+0000","last_active":"2026-02-25T15:07:50.567169+0000","last_peered":"2026-02-25T15:07:50.567169+0000","last_clean":"2026-02-25T15:07:50.567169+0000","last_became_active":"2026-02-25T15:02:15.249084+0000","last_became_peered":"2026-02-25T15:02:15.249084+0000","last_unstale":"2026-02-25T15:07:50.567169+0000","last_undegraded":"2026-02-25T15:07:50.567169+0000","last_fullsized":"2026-02-25T15:07:50.567169+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T02:14:00.305009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":621,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.563726+0000","last_change":"2026-02-25T15:02:14.838721+0000","last_active":"2026-02-25T15:07:50.563726+0000","last_peered":"2026-02-25T15:07:50.563726+0000","last_clean":"2026-02-25T15:07:50.563726+0000","last_became_active":"2026-02-25T15:02:14.836539+0000","last_became_peered":"2026-02-25T15:02:14.836539+0000","last_unstale":"2026-02-25T15:07:50.563726+0000","last_undegraded":"2026-02-25T15:07:50.563726+0000","last_fullsized":"2026-02-25T15:07:50.563726+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T02:18:20.966419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,13],"acting":[12,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":621,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.569052+0000","last_change":"2026-02-25T15:02:15.040439+0000","last_active":"2026-02-25T15:07:50.569052+0000","last_peered":"2026-02-25T15:07:50.569052+0000","last_clean":"2026-02-25T15:07:50.569052+0000","last_became_active":"2026-02-25T15:02:15.038261+0000","last_became_peered":"2026-02-25T15:02:15.038261+0000","last_unstale":"2026-02-25T15:07:50.569052+0000","last_undegraded":"2026-02-25T15:07:50.569052+0000","last_fullsized":"2026-02-25T15:07:50.569052+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:38:10.779195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,7],"acting":[12,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":619,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.561849+0000","last_change":"2026-02-25T15:02:13.433671+0000","last_active":"2026-02-25T15:07:49.561849+0000","last_peered":"2026-02-25T15:07:49.561849+0000","last_clean":"2026-02-25T15:07:49.561849+0000","last_became_active":"2026-02-25T15:02:13.426671+0000","last_became_peered":"2026-02-25T15:02:13.426671+0000","last_unstale":"2026-02-25T15:07:49.561849+0000","last_undegraded":"2026-02-25T15:07:49.561849+0000","last_fullsized":"2026-02-25T15:07:49.561849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:54:40.464012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10],"acting":[8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.571571+0000","last_change":"2026-02-25T15:02:14.942239+0000","last_active":"2026-02-25T15:07:50.571571+0000","last_peered":"2026-02-25T15:07:50.571571+0000","last_clean":"2026-02-25T15:07:50.571571+0000","last_became_active":"2026-02-25T15:02:14.934430+0000","last_became_peered":"2026-02-25T15:02:14.934430+0000","last_unstale":"2026-02-25T15:07:50.571571+0000","last_undegraded":"2026-02-25T15:07:50.571571+0000","last_fullsized":"2026-02-25T15:07:50.571571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:23:42.860020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8],"acting":[9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":621,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.565359+0000","last_change":"2026-02-25T15:02:14.166407+0000","last_active":"2026-02-25T15:07:50.565359+0000","last_peered":"2026-02-25T15:07:50.565359+0000","last_clean":"2026-02-25T15:07:50.565359+0000","last_became_active":"2026-02-25T15:02:14.164238+0000","last_became_peered":"2026-02-25T15:02:14.164238+0000","last_unstale":"2026-02-25T15:07:50.565359+0000","last_undegraded":"2026-02-25T15:07:50.565359+0000","last_fullsized":"2026-02-25T15:07:50.565359+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:06:11.961410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,15],"acting":[12,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.565158+0000","last_change":"2026-02-25T15:02:15.082986+0000","last_active":"2026-02-25T15:07:50.565158+0000","last_peered":"2026-02-25T15:07:50.565158+0000","last_clean":"2026-02-25T15:07:50.565158+0000","last_became_active":"2026-02-25T15:02:15.078177+0000","last_became_peered":"2026-02-25T15:02:15.078177+0000","last_unstale":"2026-02-25T15:07:50.565158+0000","last_undegraded":"2026-02-25T15:07:50.565158+0000","last_fullsized":"2026-02-25T15:07:50.565158+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:27:42.619819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10],"acting":[9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.563730+0000","last_change":"2026-02-25T15:02:14.239902+0000","last_active":"2026-02-25T15:07:50.563730+0000","last_peered":"2026-02-25T15:07:50.563730+0000","last_clean":"2026-02-25T15:07:50.563730+0000","last_became_active":"2026-02-25T15:02:14.237082+0000","last_became_peered":"2026-02-25T15:02:14.237082+0000","last_unstale":"2026-02-25T15:07:50.563730+0000","last_undegraded":"2026-02-25T15:07:50.563730+0000","last_fullsized":"2026-02-25T15:07:50.563730+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:57:36.384884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,14],"acting":[9,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.437951+0000","last_change":"2026-02-25T15:02:14.132167+0000","last_active":"2026-02-25T15:07:48.437951+0000","last_peered":"2026-02-25T15:07:48.437951+0000","last_clean":"2026-02-25T15:07:48.437951+0000","last_became_active":"2026-02-25T15:02:14.116657+0000","last_became_peered":"2026-02-25T15:02:14.116657+0000","last_unstale":"2026-02-25T15:07:48.437951+0000","last_undegraded":"2026-02-25T15:07:48.437951+0000","last_fullsized":"2026-02-25T15:07:48.437951+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:23:36.296106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":16}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280591,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33644,"kb_used_data":6684,"kb_used_omap":5740,"kb_used_meta":21203,"kb_avail":94338196,"statfs":{"total":96636764160,"available":96602312704,"internally_reserved":0,"allocated":6844416,"data_stored":4222855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5878530,"internal_metadata":21712126},"hb_peers":[0,1,3,4,5,6,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280591,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6676,"kb_used_omap":203,"kb_used_meta":26740,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6836224,"data_stored":4222831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":208491,"internal_metadata":27382165},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280590,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34088,"kb_used_data":7128,"kb_used_omap":203,"kb_used_meta":26740,"kb_avail":94337752,"statfs":{"total":96636764160,"available":96601858048,"internally_reserved":0,"allocated":7299072,"data_stored":4682111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":208377,"internal_metadata":27382279},"hb_peers":[1,2,4,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313295,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51668,"kb_used_data":6660,"kb_used_omap":7601,"kb_used_meta":37390,"kb_avail":94320172,"statfs":{"total":96636764160,"available":96583856128,"internally_reserved":0,"allocated":6819840,"data_stored":4212890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7784319,"internal_metadata":38287489},"hb_peers":[0,2,3,4,5,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":14,"up_from":21,"seq":90194313294,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33604,"kb_used_data":6644,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338236,"statfs":{"total":96636764160,"available":96602353664,"internally_reserved":0,"allocated":6803456,"data_stored":4200771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210523,"internal_metadata":27380133},"hb_peers":[0,1,2,3,4,5,6,8,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313294,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33620,"kb_used_data":6660,"kb_used_omap":203,"kb_used_meta":26740,"kb_avail":94338220,"statfs":{"total":96636764160,"available":96602337280,"internally_reserved":0,"allocated":6819840,"data_stored":4212890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":207913,"internal_metadata":27382743},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313294,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33620,"kb_used_data":6660,"kb_used_omap":204,"kb_used_meta":26739,"kb_avail":94338220,"statfs":{"total":96636764160,"available":96602337280,"internally_reserved":0,"allocated":6819840,"data_stored":4212890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":209288,"internal_metadata":27381368},"hb_peers":[1,2,3,4,5,6,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313294,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33620,"kb_used_data":6660,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338220,"statfs":{"total":96636764160,"available":96602337280,"internally_reserved":0,"allocated":6819840,"data_stored":4212890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":209978,"internal_metadata":27380678},"hb_peers":[1,3,5,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313294,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33620,"kb_used_data":6660,"kb_used_omap":204,"kb_used_meta":26739,"kb_avail":94338220,"statfs":{"total":96636764160,"available":96602337280,"internally_reserved":0,"allocated":6819840,"data_stored":4212890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":209789,"internal_metadata":27380867},"hb_peers":[0,1,2,3,4,5,6,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313295,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6676,"kb_used_omap":203,"kb_used_meta":26740,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6836224,"data_stored":4222831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":208876,"internal_metadata":27381780},"hb_peers":[1,3,5,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313294,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6676,"kb_used_omap":206,"kb_used_meta":26737,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6836224,"data_stored":4222831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":211548,"internal_metadata":27379108},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313295,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34088,"kb_used_data":7128,"kb_used_omap":206,"kb_used_meta":26737,"kb_avail":94337752,"statfs":{"total":96636764160,"available":96601858048,"internally_reserved":0,"allocated":7299072,"data_stored":4682111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":211268,"internal_metadata":27379388},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313294,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6676,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6836224,"data_stored":4222831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210167,"internal_metadata":27380489},"hb_peers":[0,2,4,5,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313294,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33624,"kb_used_data":6664,"kb_used_omap":5825,"kb_used_meta":21118,"kb_avail":94338216,"statfs":{"total":96636764160,"available":96602333184,"internally_reserved":0,"allocated":6823936,"data_stored":4212895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5965730,"internal_metadata":21624926},"hb_peers":[0,1,2,4,5,6,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313295,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33624,"kb_used_data":6664,"kb_used_omap":203,"kb_used_meta":26740,"kb_avail":94338216,"statfs":{"total":96636764160,"available":96602333184,"internally_reserved":0,"allocated":6823936,"data_stored":4212909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":208021,"internal_metadata":27382635},"hb_peers":[0,1,2,3,4,5,7,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313295,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6676,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6836224,"data_stored":4222831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210527,"internal_metadata":27380129},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T15:07:52.455 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T15:07:52.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.520+0000 7fb2d3fb1640 10 monclient: get_monmap_and_config 2026-02-25T15:07:52.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.520+0000 7fb2d3fb1640 10 monclient: build_initial_monmap 2026-02-25T15:07:52.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.521+0000 7fb2d3fb1640 10 monclient: monmap: 2026-02-25T15:07:52.523 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:52.524 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.524 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:52.522638+0000 2026-02-25T15:07:52.524 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:52.522638+0000 2026-02-25T15:07:52.524 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:52.524 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.524 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:52.524 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:52.524 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:52.524 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.521+0000 7fb2d3fb1640 1 Processor -- start 2026-02-25T15:07:52.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.521+0000 7fb2d3fb1640 1 -- start start 2026-02-25T15:07:52.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.521+0000 7fb2d3fb1640 10 monclient: init 2026-02-25T15:07:52.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.521+0000 7fb2d3fb1640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:52.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.521+0000 7fb2d3fb1640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:07:52.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.521+0000 7fb2d3fb1640 10 monclient(hunting): picked mon.noname-a con 0x7fb2cc058da0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:52.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.521+0000 7fb2d3fb1640 10 monclient(hunting): picked mon.noname-b con 0x7fb2cc0b0410 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:52.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.521+0000 7fb2d3fb1640 10 monclient(hunting): picked mon.noname-c con 0x7fb2cc1626a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:52.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.521+0000 7fb2d3fb1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2cc1655c0 con 0x7fb2cc058da0 2026-02-25T15:07:52.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.521+0000 7fb2d3fb1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2cc0b12b0 con 0x7fb2cc0b0410 2026-02-25T15:07:52.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.521+0000 7fb2d3fb1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2cc0b1590 con 0x7fb2cc1626a0 2026-02-25T15:07:52.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.521+0000 7fb2d3fb1640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:52.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.522+0000 7fb2d3fb1640 10 monclient(hunting): authenticate will time out at 3362.123003s 2026-02-25T15:07:52.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.522+0000 7fb2d2527640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7fb2cc1626a0 0x7fb2cc1c6d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:36666/0 (socket says 10.20.193.90:36666) 2026-02-25T15:07:52.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.522+0000 7fb2d2527640 1 -- 10.20.193.90:0/679140438 learned_addr learned my addr 10.20.193.90:0/679140438 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:52.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.523+0000 7fb2d0d24640 1 -- 10.20.193.90:0/679140438 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787436281 0 0) 0x7fb2cc0b1590 con 0x7fb2cc1626a0 2026-02-25T15:07:52.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.523+0000 7fb2d0d24640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.523+0000 7fb2d0d24640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.523+0000 7fb2d0d24640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.523+0000 7fb2d0d24640 10 monclient(hunting): my global_id is 28535 2026-02-25T15:07:52.527 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.523+0000 7fb2d0d24640 1 -- 10.20.193.90:0/679140438 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2b4003610 con 0x7fb2cc1626a0 2026-02-25T15:07:52.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.523+0000 7fb2d0d24640 1 -- 10.20.193.90:0/679140438 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2195105689 0 0) 0x7fb2cc0b12b0 con 0x7fb2cc0b0410 2026-02-25T15:07:52.532 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.523+0000 7fb2d0d24640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.523+0000 7fb2d0d24640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.523+0000 7fb2d0d24640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.523+0000 7fb2d0d24640 10 monclient(hunting): my global_id is 27397 2026-02-25T15:07:52.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.523+0000 7fb2d0d24640 1 -- 10.20.193.90:0/679140438 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2cc0b1590 con 0x7fb2cc0b0410 2026-02-25T15:07:52.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.524+0000 7fb2d0d24640 1 -- 10.20.193.90:0/679140438 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2708090418 0 0) 0x7fb2cc1655c0 con 0x7fb2cc058da0 2026-02-25T15:07:52.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.524+0000 7fb2d0d24640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.524+0000 7fb2d0d24640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.524+0000 7fb2d0d24640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.524+0000 7fb2d0d24640 10 monclient(hunting): my global_id is 30765 2026-02-25T15:07:52.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.524+0000 7fb2d0d24640 1 -- 10.20.193.90:0/679140438 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2cc0b12b0 con 0x7fb2cc058da0 2026-02-25T15:07:52.533 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d0d24640 1 -- 10.20.193.90:0/679140438 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2498823421 0 0) 0x7fb2b4003610 con 0x7fb2cc1626a0 2026-02-25T15:07:52.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d0d24640 1 -- 10.20.193.90:0/679140438 >> v1:10.20.193.93:6789/0 conn(0x7fb2cc0b0410 legacy=0x7fb2cc164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d0d24640 1 -- 10.20.193.90:0/679140438 >> v1:10.20.193.90:6789/0 conn(0x7fb2cc058da0 legacy=0x7fb2cc1647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d0d24640 10 monclient: _finish_hunting 0 2026-02-25T15:07:52.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d0d24640 1 monclient: found mon.noname-c 2026-02-25T15:07:52.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d0d24640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:52.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d0d24640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:07:52.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d0d24640 1 -- 10.20.193.90:0/679140438 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2cc0b1870 con 0x7fb2cc1626a0 2026-02-25T15:07:52.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d0d24640 10 monclient: _finish_auth 0 2026-02-25T15:07:52.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d0d24640 10 monclient: _check_auth_tickets 2026-02-25T15:07:52.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d0d24640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:52.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d0d24640 1 -- 10.20.193.90:0/679140438 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb2c8003170 con 0x7fb2cc1626a0 2026-02-25T15:07:52.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d0d24640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:52.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d0d24640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:07:52.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d0d24640 10 monclient: dump: 2026-02-25T15:07:52.535 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.535 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.535 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.535 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.535 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.535 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.535 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.535 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.535 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.535 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d3fb1640 5 monclient: authenticate success, global_id 28535 2026-02-25T15:07:52.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.525+0000 7fb2d3fb1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:52.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.526+0000 7fb2d0d24640 1 -- 10.20.193.90:0/679140438 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb2c8003340 con 0x7fb2cc1626a0 2026-02-25T15:07:52.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.526+0000 7fb2d0d24640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:52.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.526+0000 7fb2d0d24640 1 -- 10.20.193.90:0/679140438 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb2c8003780 con 0x7fb2cc1626a0 2026-02-25T15:07:52.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.526+0000 7fb2d0d24640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:52.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.526+0000 7fb2d0d24640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:52.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.526+0000 7fb2d0d24640 10 monclient: dump: 2026-02-25T15:07:52.536 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.536 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.536 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.536 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.536 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.537 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.537 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.537 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.537 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.537 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.526+0000 7fb2d3fb1640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:52.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.526+0000 7fb2d3fb1640 4 set_mon_vals no callback set 2026-02-25T15:07:52.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.526+0000 7fb2d3fb1640 10 monclient: shutdown 2026-02-25T15:07:52.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.526+0000 7fb2d3fb1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:52.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.526+0000 7fb2d3fb1640 1 -- 10.20.193.90:0/679140438 >> v1:10.20.193.94:6789/0 conn(0x7fb2cc1626a0 legacy=0x7fb2cc1c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.526+0000 7fb2d3fb1640 1 -- 10.20.193.90:0/679140438 shutdown_connections 2026-02-25T15:07:52.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.526+0000 7fb2d3fb1640 1 -- 10.20.193.90:0/679140438 >> 10.20.193.90:0/679140438 conn(0x7fb2cc0b65f0 msgr2=0x7fb2cc0b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:52.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.526+0000 7fb2d3fb1640 1 -- 10.20.193.90:0/679140438 shutdown_connections 2026-02-25T15:07:52.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.526+0000 7fb2d3fb1640 1 -- 10.20.193.90:0/679140438 wait complete. 2026-02-25T15:07:52.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 10 monclient: build_initial_monmap 2026-02-25T15:07:52.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 10 monclient: monmap: 2026-02-25T15:07:52.538 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:52.538 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:52.538 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:52.538 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:52.538 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:52.538 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:52.538 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:52.538 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:52.539 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:52.539 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 1 librados: starting msgr at 2026-02-25T15:07:52.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 1 librados: starting objecter 2026-02-25T15:07:52.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:52.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 1 Processor -- start 2026-02-25T15:07:52.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 1 -- start start 2026-02-25T15:07:52.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 1 librados: setting wanted keys 2026-02-25T15:07:52.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 1 librados: calling monclient init 2026-02-25T15:07:52.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 10 monclient: init 2026-02-25T15:07:52.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:52.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:07:52.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 10 monclient(hunting): picked mon.noname-c con 0x7fb2cc058da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:52.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 10 monclient(hunting): picked mon.noname-b con 0x7fb2cc0b0410 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:52.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 10 monclient(hunting): picked mon.noname-a con 0x7fb2cc1626a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:52.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2cc1fcc80 con 0x7fb2cc1626a0 2026-02-25T15:07:52.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2cc1fc6c0 con 0x7fb2cc0b0410 2026-02-25T15:07:52.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2cc1fc9a0 con 0x7fb2cc058da0 2026-02-25T15:07:52.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:52.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.527+0000 7fb2d3fb1640 10 monclient(hunting): authenticate will time out at 3362.128773s 2026-02-25T15:07:52.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.528+0000 7fb2d1525640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fb2cc0b0410 0x7fb2cc156940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:34106/0 (socket says 10.20.193.90:34106) 2026-02-25T15:07:52.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.528+0000 7fb2d1525640 1 -- 10.20.193.90:0/2012265323 learned_addr learned my addr 10.20.193.90:0/2012265323 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:52.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.528+0000 7fb2ba7fc640 10 client.?.objecter ms_handle_connect 0x7fb2cc1626a0 2026-02-25T15:07:52.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.528+0000 7fb2ba7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:52.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.528+0000 7fb2ba7fc640 10 client.?.objecter ms_handle_connect 0x7fb2cc058da0 2026-02-25T15:07:52.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.528+0000 7fb2ba7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:52.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.528+0000 7fb2ba7fc640 10 client.?.objecter ms_handle_connect 0x7fb2cc0b0410 2026-02-25T15:07:52.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.528+0000 7fb2ba7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:52.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 641111054 0 0) 0x7fb2cc1fcc80 con 0x7fb2cc1626a0 2026-02-25T15:07:52.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 10 monclient(hunting): my global_id is 30768 2026-02-25T15:07:52.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb29c003610 con 0x7fb2cc1626a0 2026-02-25T15:07:52.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1291728196 0 0) 0x7fb2cc1fc9a0 con 0x7fb2cc058da0 2026-02-25T15:07:52.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 10 monclient(hunting): my global_id is 28538 2026-02-25T15:07:52.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2cc1fcc80 con 0x7fb2cc058da0 2026-02-25T15:07:52.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1329589731 0 0) 0x7fb2cc1fc6c0 con 0x7fb2cc0b0410 2026-02-25T15:07:52.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 10 monclient(hunting): my global_id is 27400 2026-02-25T15:07:52.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.529+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2cc1fc9a0 con 0x7fb2cc0b0410 2026-02-25T15:07:52.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.531+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3682931235 0 0) 0x7fb29c003610 con 0x7fb2cc1626a0 2026-02-25T15:07:52.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.531+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb2cc1fc6c0 con 0x7fb2cc1626a0 2026-02-25T15:07:52.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.531+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb2c8003690 con 0x7fb2cc1626a0 2026-02-25T15:07:52.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.531+0000 7fb2ba7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:52.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.531+0000 7fb2ba7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:52.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.531+0000 7fb2ba7fc640 10 monclient(hunting): dump: 2026-02-25T15:07:52.545 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.545 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.545 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.545 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.546 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.546 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.546 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.546 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.546 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.546 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.531+0000 7fb2ba7fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:52.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.531+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 689951676 0 0) 0x7fb2cc1fcc80 con 0x7fb2cc058da0 2026-02-25T15:07:52.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.531+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb29c003610 con 0x7fb2cc058da0 2026-02-25T15:07:52.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.531+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb2bc003060 con 0x7fb2cc058da0 2026-02-25T15:07:52.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.531+0000 7fb2ba7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:52.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.531+0000 7fb2ba7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:52.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.531+0000 7fb2ba7fc640 10 monclient(hunting): dump: 2026-02-25T15:07:52.547 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.547 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.547 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.547 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.547 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.547 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.547 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.547 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.547 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.547 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.532+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 301326678 0 0) 0x7fb2cc1fc9a0 con 0x7fb2cc0b0410 2026-02-25T15:07:52.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.532+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb2cc1fcc80 con 0x7fb2cc0b0410 2026-02-25T15:07:52.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.532+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb2c0003170 con 0x7fb2cc0b0410 2026-02-25T15:07:52.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.532+0000 7fb2ba7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:52.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.532+0000 7fb2ba7fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:52.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.532+0000 7fb2ba7fc640 10 monclient(hunting): dump: 2026-02-25T15:07:52.548 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.548 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.548 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.548 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.548 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.548 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.548 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.549 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.549 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.549 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3187955054 0 0) 0x7fb2cc1fc6c0 con 0x7fb2cc1626a0 2026-02-25T15:07:52.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 >> v1:10.20.193.94:6789/0 conn(0x7fb2cc058da0 legacy=0x7fb2cc1fbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 >> v1:10.20.193.93:6789/0 conn(0x7fb2cc0b0410 legacy=0x7fb2cc156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2ba7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:07:52.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2ba7fc640 1 monclient: found mon.a 2026-02-25T15:07:52.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2ba7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:52.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2ba7fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:52.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2cc1f88a0 con 0x7fb2cc1626a0 2026-02-25T15:07:52.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2ba7fc640 10 monclient: _finish_auth 0 2026-02-25T15:07:52.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2ba7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:07:52.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:52.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2d3fb1640 5 monclient: authenticate success, global_id 30768 2026-02-25T15:07:52.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2d3fb1640 10 monclient: _renew_subs 2026-02-25T15:07:52.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2d3fb1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:52.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2d3fb1640 1 -- 10.20.193.90:0/2012265323 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb2cc1f8b80 con 0x7fb2cc1626a0 2026-02-25T15:07:52.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2d3fb1640 10 client.30768.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:52.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2d3fb1640 10 monclient: _renew_subs 2026-02-25T15:07:52.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2d3fb1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:52.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2d3fb1640 1 -- 10.20.193.90:0/2012265323 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb2cc157050 con 0x7fb2cc1626a0 2026-02-25T15:07:52.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2d3fb1640 1 librados: init done 2026-02-25T15:07:52.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2d3fb1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:52.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2d3fb1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:52.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2d3fb1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:52.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.533+0000 7fb2d3fb1640 1 -- 10.20.193.90:0/2012265323 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2cc0b6d90 con 0x7fb2cc1626a0 2026-02-25T15:07:52.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.623+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb2c8003300 con 0x7fb2cc1626a0 2026-02-25T15:07:52.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.623+0000 7fb2ba7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:52.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.623+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb2c8003830 con 0x7fb2cc1626a0 2026-02-25T15:07:52.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.623+0000 7fb2ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:52.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.623+0000 7fb2ba7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:52.626 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.623+0000 7fb2ba7fc640 10 monclient: dump: 2026-02-25T15:07:52.627 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.627 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.627 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.627 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.627 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.627 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.627 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.627 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.627 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.627 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.627 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.623+0000 7fb2bbfff640 4 set_mon_vals no callback set 2026-02-25T15:07:52.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.624+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fb2c801e240 con 0x7fb2cc1626a0 2026-02-25T15:07:52.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.625+0000 7fb2d1d26640 10 monclient: get_auth_request con 0x7fb29c07a040 auth_method 0 2026-02-25T15:07:52.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.625+0000 7fb2ba7fc640 10 client.30768.objecter ms_handle_connect 0x7fb29c07a040 2026-02-25T15:07:52.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.626+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(337..337 src has 1..337) ==== 8959+0+0 (unknown 3556021173 0 0) 0x7fb2c8095b80 con 0x7fb2cc1626a0 2026-02-25T15:07:52.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.626+0000 7fb2ba7fc640 10 client.30768.objecter ms_dispatch2 0x7fb2cc000b90 osd_map(337..337 src has 1..337) 2026-02-25T15:07:52.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.626+0000 7fb2ba7fc640 3 client.30768.objecter handle_osd_map got epochs [337,337] > 0 2026-02-25T15:07:52.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.626+0000 7fb2ba7fc640 3 client.30768.objecter handle_osd_map decoding full epoch 337 2026-02-25T15:07:52.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.626+0000 7fb2ba7fc640 20 client.30768.objecter dump_active .. 0 homeless 2026-02-25T15:07:52.629 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.626+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fb2c805f4e0 con 0x7fb2cc1626a0 2026-02-25T15:07:52.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.626+0000 7fb2ba7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:52.630 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.626+0000 7fb2ba7fc640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:52.737 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.734+0000 7fb2d3fb1640 1 -- 10.20.193.90:0/2012265323 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb2cc0b2780 con 0x7fb29c07a040 2026-02-25T15:07:52.738 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.736+0000 7fb2ba7fc640 1 -- 10.20.193.90:0/2012265323 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+125395 (unknown 2965378022 0 1250104598) 0x7fb2cc0b2780 con 0x7fb29c07a040 2026-02-25T15:07:52.739 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:07:52.740 INFO:teuthology.orchestra.run.trial090.stderr:dumped all 2026-02-25T15:07:52.742 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.737+0000 7fb2d3fb1640 10 librados: watch_flush enter 2026-02-25T15:07:52.742 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.737+0000 7fb2d3fb1640 10 librados: watch_flush exit 2026-02-25T15:07:52.742 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.737+0000 7fb2d3fb1640 20 client.30768.objecter shutdown clearing up homeless session... 2026-02-25T15:07:52.742 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.737+0000 7fb2d3fb1640 10 client.30768.objecter successfully canceled tick 2026-02-25T15:07:52.742 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.737+0000 7fb2d3fb1640 1 -- 10.20.193.90:0/2012265323 >> v1:10.20.193.189:6800/3647375982 conn(0x7fb29c07a040 legacy=0x7fb29c09a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.742 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.737+0000 7fb2d3fb1640 10 monclient: shutdown 2026-02-25T15:07:52.742 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.737+0000 7fb2d3fb1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:52.742 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.737+0000 7fb2d3fb1640 1 -- 10.20.193.90:0/2012265323 >> v1:10.20.193.90:6789/0 conn(0x7fb2cc1626a0 legacy=0x7fb2cc164430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.743 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.737+0000 7fb2d3fb1640 1 -- 10.20.193.90:0/2012265323 shutdown_connections 2026-02-25T15:07:52.743 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.737+0000 7fb2d3fb1640 1 -- 10.20.193.90:0/2012265323 >> 10.20.193.90:0/2012265323 conn(0x7fb2cc0b65f0 msgr2=0x7fb2cc13c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:52.743 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.737+0000 7fb2d3fb1640 1 -- 10.20.193.90:0/2012265323 shutdown_connections 2026-02-25T15:07:52.743 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.737+0000 7fb2d3fb1640 1 -- 10.20.193.90:0/2012265323 wait complete. 2026-02-25T15:07:52.743 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.737+0000 7fb2d3fb1640 1 librados: shutdown 2026-02-25T15:07:52.748 INFO:teuthology.orchestra.run.trial090.stdout:{"pg_ready":true,"pg_map":{"version":530,"stamp":"2026-02-25T15:07:51.593123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":56,"num_read_kb":47,"num_write":70,"num_write_kb":597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":557000,"kb_used_data":107592,"kb_used_omap":21829,"kb_used_meta":427322,"kb_avail":1509392440,"statfs":{"total":1546188226560,"available":1545617858560,"internally_reserved":0,"allocated":110174208,"data_stored":68392257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22353345,"internal_metadata":437578303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":78,"apply_latency_ms":78,"commit_latency_ns":78000000,"apply_latency_ns":78000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.158859"},"pg_stats":[{"pgid":"25.5","version":"0'0","reported_seq":619,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.437533+0000","last_change":"2026-02-25T15:02:14.833079+0000","last_active":"2026-02-25T15:07:48.437533+0000","last_peered":"2026-02-25T15:07:48.437533+0000","last_clean":"2026-02-25T15:07:48.437533+0000","last_became_active":"2026-02-25T15:02:14.830708+0000","last_became_peered":"2026-02-25T15:02:14.830708+0000","last_unstale":"2026-02-25T15:07:48.437533+0000","last_undegraded":"2026-02-25T15:07:48.437533+0000","last_fullsized":"2026-02-25T15:07:48.437533+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:21:07.436552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,8],"acting":[14,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.569659+0000","last_change":"2026-02-25T15:02:14.703677+0000","last_active":"2026-02-25T15:07:50.569659+0000","last_peered":"2026-02-25T15:07:50.569659+0000","last_clean":"2026-02-25T15:07:50.569659+0000","last_became_active":"2026-02-25T15:02:14.686130+0000","last_became_peered":"2026-02-25T15:02:14.686130+0000","last_unstale":"2026-02-25T15:07:50.569659+0000","last_undegraded":"2026-02-25T15:07:50.569659+0000","last_fullsized":"2026-02-25T15:07:50.569659+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:37:58.016312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7],"acting":[9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.444074+0000","last_change":"2026-02-25T15:02:14.184346+0000","last_active":"2026-02-25T15:07:48.444074+0000","last_peered":"2026-02-25T15:07:48.444074+0000","last_clean":"2026-02-25T15:07:48.444074+0000","last_became_active":"2026-02-25T15:02:14.182253+0000","last_became_peered":"2026-02-25T15:02:14.182253+0000","last_unstale":"2026-02-25T15:07:48.444074+0000","last_undegraded":"2026-02-25T15:07:48.444074+0000","last_fullsized":"2026-02-25T15:07:48.444074+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:10:33.826830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.442617+0000","last_change":"2026-02-25T15:02:14.262914+0000","last_active":"2026-02-25T15:07:48.442617+0000","last_peered":"2026-02-25T15:07:48.442617+0000","last_clean":"2026-02-25T15:07:48.442617+0000","last_became_active":"2026-02-25T15:02:14.258456+0000","last_became_peered":"2026-02-25T15:02:14.258456+0000","last_unstale":"2026-02-25T15:07:48.442617+0000","last_undegraded":"2026-02-25T15:07:48.442617+0000","last_fullsized":"2026-02-25T15:07:48.442617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:29:58.464087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"33'11","reported_seq":645,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.505217+0000","last_change":"2026-02-25T15:02:15.833921+0000","last_active":"2026-02-25T15:07:49.505217+0000","last_peered":"2026-02-25T15:07:49.505217+0000","last_clean":"2026-02-25T15:07:49.505217+0000","last_became_active":"2026-02-25T15:02:15.819620+0000","last_became_peered":"2026-02-25T15:02:15.819620+0000","last_unstale":"2026-02-25T15:07:49.505217+0000","last_undegraded":"2026-02-25T15:07:49.505217+0000","last_fullsized":"2026-02-25T15:07:49.505217+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:40:10.738663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,13],"acting":[10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":616,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.562854+0000","last_change":"2026-02-25T15:02:13.775077+0000","last_active":"2026-02-25T15:07:49.562854+0000","last_peered":"2026-02-25T15:07:49.562854+0000","last_clean":"2026-02-25T15:07:49.562854+0000","last_became_active":"2026-02-25T15:02:13.773022+0000","last_became_peered":"2026-02-25T15:02:13.773022+0000","last_unstale":"2026-02-25T15:07:49.562854+0000","last_undegraded":"2026-02-25T15:07:49.562854+0000","last_fullsized":"2026-02-25T15:07:49.562854+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:04:02.164409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":620,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.558382+0000","last_change":"2026-02-25T15:02:14.036683+0000","last_active":"2026-02-25T15:07:49.558382+0000","last_peered":"2026-02-25T15:07:49.558382+0000","last_clean":"2026-02-25T15:07:49.558382+0000","last_became_active":"2026-02-25T15:02:14.028952+0000","last_became_peered":"2026-02-25T15:02:14.028952+0000","last_unstale":"2026-02-25T15:07:49.558382+0000","last_undegraded":"2026-02-25T15:07:49.558382+0000","last_fullsized":"2026-02-25T15:07:49.558382+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:24:37.223453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,13],"acting":[5,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":619,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.584196+0000","last_change":"2026-02-25T15:02:14.938617+0000","last_active":"2026-02-25T15:07:50.584196+0000","last_peered":"2026-02-25T15:07:50.584196+0000","last_clean":"2026-02-25T15:07:50.584196+0000","last_became_active":"2026-02-25T15:02:14.936242+0000","last_became_peered":"2026-02-25T15:02:14.936242+0000","last_unstale":"2026-02-25T15:07:50.584196+0000","last_undegraded":"2026-02-25T15:07:50.584196+0000","last_fullsized":"2026-02-25T15:07:50.584196+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:59:06.072509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.579130+0000","last_change":"2026-02-25T15:02:13.982149+0000","last_active":"2026-02-25T15:07:48.579130+0000","last_peered":"2026-02-25T15:07:48.579130+0000","last_clean":"2026-02-25T15:07:48.579130+0000","last_became_active":"2026-02-25T15:02:13.976916+0000","last_became_peered":"2026-02-25T15:02:13.976916+0000","last_unstale":"2026-02-25T15:07:48.579130+0000","last_undegraded":"2026-02-25T15:07:48.579130+0000","last_fullsized":"2026-02-25T15:07:48.579130+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:19:44.345322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":616,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.565247+0000","last_change":"2026-02-25T15:02:13.384676+0000","last_active":"2026-02-25T15:07:49.565247+0000","last_peered":"2026-02-25T15:07:49.565247+0000","last_clean":"2026-02-25T15:07:49.565247+0000","last_became_active":"2026-02-25T15:02:13.382450+0000","last_became_peered":"2026-02-25T15:02:13.382450+0000","last_unstale":"2026-02-25T15:07:49.565247+0000","last_undegraded":"2026-02-25T15:07:49.565247+0000","last_fullsized":"2026-02-25T15:07:49.565247+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:23:31.158323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10],"acting":[0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":617,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.605481+0000","last_change":"2026-02-25T15:02:13.940723+0000","last_active":"2026-02-25T15:07:49.605481+0000","last_peered":"2026-02-25T15:07:49.605481+0000","last_clean":"2026-02-25T15:07:49.605481+0000","last_became_active":"2026-02-25T15:02:13.934323+0000","last_became_peered":"2026-02-25T15:02:13.934323+0000","last_unstale":"2026-02-25T15:07:49.605481+0000","last_undegraded":"2026-02-25T15:07:49.605481+0000","last_fullsized":"2026-02-25T15:07:49.605481+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:14:02.898606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10],"acting":[11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":620,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.567595+0000","last_change":"2026-02-25T15:02:14.071335+0000","last_active":"2026-02-25T15:07:49.567595+0000","last_peered":"2026-02-25T15:07:49.567595+0000","last_clean":"2026-02-25T15:07:49.567595+0000","last_became_active":"2026-02-25T15:02:14.063362+0000","last_became_peered":"2026-02-25T15:02:14.063362+0000","last_unstale":"2026-02-25T15:07:49.567595+0000","last_undegraded":"2026-02-25T15:07:49.567595+0000","last_fullsized":"2026-02-25T15:07:49.567595+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:41:34.565838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,14],"acting":[5,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.440309+0000","last_change":"2026-02-25T15:02:13.851333+0000","last_active":"2026-02-25T15:07:48.440309+0000","last_peered":"2026-02-25T15:07:48.440309+0000","last_clean":"2026-02-25T15:07:48.440309+0000","last_became_active":"2026-02-25T15:02:13.849038+0000","last_became_peered":"2026-02-25T15:02:13.849038+0000","last_unstale":"2026-02-25T15:07:48.440309+0000","last_undegraded":"2026-02-25T15:07:48.440309+0000","last_fullsized":"2026-02-25T15:07:48.440309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:04:11.634095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":632,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.604755+0000","last_change":"2026-02-25T15:01:40.617020+0000","last_active":"2026-02-25T15:07:49.604755+0000","last_peered":"2026-02-25T15:07:49.604755+0000","last_clean":"2026-02-25T15:07:49.604755+0000","last_became_active":"2026-02-25T15:01:38.997857+0000","last_became_peered":"2026-02-25T15:01:38.997857+0000","last_unstale":"2026-02-25T15:07:49.604755+0000","last_undegraded":"2026-02-25T15:07:49.604755+0000","last_fullsized":"2026-02-25T15:07:49.604755+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:49:56.506302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062975700999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14],"acting":[11,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":617,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.546518+0000","last_change":"2026-02-25T15:02:13.715917+0000","last_active":"2026-02-25T15:07:49.546518+0000","last_peered":"2026-02-25T15:07:49.546518+0000","last_clean":"2026-02-25T15:07:49.546518+0000","last_became_active":"2026-02-25T15:02:13.713599+0000","last_became_peered":"2026-02-25T15:02:13.713599+0000","last_unstale":"2026-02-25T15:07:49.546518+0000","last_undegraded":"2026-02-25T15:07:49.546518+0000","last_fullsized":"2026-02-25T15:07:49.546518+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:15:39.798144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":636,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.567836+0000","last_change":"2026-02-25T15:01:41.128386+0000","last_active":"2026-02-25T15:07:50.567836+0000","last_peered":"2026-02-25T15:07:50.567836+0000","last_clean":"2026-02-25T15:07:50.567836+0000","last_became_active":"2026-02-25T15:01:38.782341+0000","last_became_peered":"2026-02-25T15:01:38.782341+0000","last_unstale":"2026-02-25T15:07:50.567836+0000","last_undegraded":"2026-02-25T15:07:50.567836+0000","last_fullsized":"2026-02-25T15:07:50.567836+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:07:32.765627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031008292,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,3],"acting":[12,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":619,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.563927+0000","last_change":"2026-02-25T15:02:13.858401+0000","last_active":"2026-02-25T15:07:50.563927+0000","last_peered":"2026-02-25T15:07:50.563927+0000","last_clean":"2026-02-25T15:07:50.563927+0000","last_became_active":"2026-02-25T15:02:13.846214+0000","last_became_peered":"2026-02-25T15:02:13.846214+0000","last_unstale":"2026-02-25T15:07:50.563927+0000","last_undegraded":"2026-02-25T15:07:50.563927+0000","last_fullsized":"2026-02-25T15:07:50.563927+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:30:33.308451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11],"acting":[3,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":634,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.596549+0000","last_change":"2026-02-25T15:01:41.143783+0000","last_active":"2026-02-25T15:07:50.596549+0000","last_peered":"2026-02-25T15:07:50.596549+0000","last_clean":"2026-02-25T15:07:50.596549+0000","last_became_active":"2026-02-25T15:01:38.731569+0000","last_became_peered":"2026-02-25T15:01:38.731569+0000","last_unstale":"2026-02-25T15:07:50.596549+0000","last_undegraded":"2026-02-25T15:07:50.596549+0000","last_fullsized":"2026-02-25T15:07:50.596549+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:00:25.099174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037093318,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15],"acting":[7,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":619,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.441335+0000","last_change":"2026-02-25T15:02:14.257499+0000","last_active":"2026-02-25T15:07:48.441335+0000","last_peered":"2026-02-25T15:07:48.441335+0000","last_clean":"2026-02-25T15:07:48.441335+0000","last_became_active":"2026-02-25T15:02:14.254843+0000","last_became_peered":"2026-02-25T15:02:14.254843+0000","last_unstale":"2026-02-25T15:07:48.441335+0000","last_undegraded":"2026-02-25T15:07:48.441335+0000","last_fullsized":"2026-02-25T15:07:48.441335+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:07:11.424289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2],"acting":[14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":634,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.593113+0000","last_change":"2026-02-25T15:01:41.139794+0000","last_active":"2026-02-25T15:07:50.593113+0000","last_peered":"2026-02-25T15:07:50.593113+0000","last_clean":"2026-02-25T15:07:50.593113+0000","last_became_active":"2026-02-25T15:01:38.780556+0000","last_became_peered":"2026-02-25T15:01:38.780556+0000","last_unstale":"2026-02-25T15:07:50.593113+0000","last_undegraded":"2026-02-25T15:07:50.593113+0000","last_fullsized":"2026-02-25T15:07:50.593113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:57:49.842137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041710525999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":616,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.560799+0000","last_change":"2026-02-25T15:02:14.919090+0000","last_active":"2026-02-25T15:07:49.560799+0000","last_peered":"2026-02-25T15:07:49.560799+0000","last_clean":"2026-02-25T15:07:49.560799+0000","last_became_active":"2026-02-25T15:02:14.894984+0000","last_became_peered":"2026-02-25T15:02:14.894984+0000","last_unstale":"2026-02-25T15:07:49.560799+0000","last_undegraded":"2026-02-25T15:07:49.560799+0000","last_fullsized":"2026-02-25T15:07:49.560799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:10:40.021124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":634,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.596471+0000","last_change":"2026-02-25T15:01:40.608781+0000","last_active":"2026-02-25T15:07:49.596471+0000","last_peered":"2026-02-25T15:07:49.596471+0000","last_clean":"2026-02-25T15:07:49.596471+0000","last_became_active":"2026-02-25T15:01:38.743941+0000","last_became_peered":"2026-02-25T15:01:38.743941+0000","last_unstale":"2026-02-25T15:07:49.596471+0000","last_undegraded":"2026-02-25T15:07:49.596471+0000","last_fullsized":"2026-02-25T15:07:49.596471+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:53:32.920667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.061484652000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13],"acting":[11,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":620,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.568600+0000","last_change":"2026-02-25T15:02:14.369913+0000","last_active":"2026-02-25T15:07:49.568600+0000","last_peered":"2026-02-25T15:07:49.568600+0000","last_clean":"2026-02-25T15:07:49.568600+0000","last_became_active":"2026-02-25T15:02:14.364385+0000","last_became_peered":"2026-02-25T15:02:14.364385+0000","last_unstale":"2026-02-25T15:07:49.568600+0000","last_undegraded":"2026-02-25T15:07:49.568600+0000","last_fullsized":"2026-02-25T15:07:49.568600+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:35:03.754835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":632,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.543792+0000","last_change":"2026-02-25T15:01:41.129594+0000","last_active":"2026-02-25T15:07:49.543792+0000","last_peered":"2026-02-25T15:07:49.543792+0000","last_clean":"2026-02-25T15:07:49.543792+0000","last_became_active":"2026-02-25T15:01:38.861789+0000","last_became_peered":"2026-02-25T15:01:38.861789+0000","last_unstale":"2026-02-25T15:07:49.543792+0000","last_undegraded":"2026-02-25T15:07:49.543792+0000","last_fullsized":"2026-02-25T15:07:49.543792+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:36:46.424677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032448785000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":617,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.598089+0000","last_change":"2026-02-25T15:02:15.000767+0000","last_active":"2026-02-25T15:07:49.598089+0000","last_peered":"2026-02-25T15:07:49.598089+0000","last_clean":"2026-02-25T15:07:49.598089+0000","last_became_active":"2026-02-25T15:02:14.998340+0000","last_became_peered":"2026-02-25T15:02:14.998340+0000","last_unstale":"2026-02-25T15:07:49.598089+0000","last_undegraded":"2026-02-25T15:07:49.598089+0000","last_fullsized":"2026-02-25T15:07:49.598089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:06:28.044352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5],"acting":[11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":632,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.562031+0000","last_change":"2026-02-25T15:01:41.116971+0000","last_active":"2026-02-25T15:07:49.562031+0000","last_peered":"2026-02-25T15:07:49.562031+0000","last_clean":"2026-02-25T15:07:49.562031+0000","last_became_active":"2026-02-25T15:01:38.737942+0000","last_became_peered":"2026-02-25T15:01:38.737942+0000","last_unstale":"2026-02-25T15:07:49.562031+0000","last_undegraded":"2026-02-25T15:07:49.562031+0000","last_fullsized":"2026-02-25T15:07:49.562031+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:51:33.138764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031295004000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13],"acting":[0,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":619,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.570012+0000","last_change":"2026-02-25T15:02:13.590713+0000","last_active":"2026-02-25T15:07:50.570012+0000","last_peered":"2026-02-25T15:07:50.570012+0000","last_clean":"2026-02-25T15:07:50.570012+0000","last_became_active":"2026-02-25T15:02:13.588299+0000","last_became_peered":"2026-02-25T15:02:13.588299+0000","last_unstale":"2026-02-25T15:07:50.570012+0000","last_undegraded":"2026-02-25T15:07:50.570012+0000","last_fullsized":"2026-02-25T15:07:50.570012+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:10:21.612478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":633,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.598833+0000","last_change":"2026-02-25T15:01:40.614625+0000","last_active":"2026-02-25T15:07:49.598833+0000","last_peered":"2026-02-25T15:07:49.598833+0000","last_clean":"2026-02-25T15:07:49.598833+0000","last_became_active":"2026-02-25T15:01:38.791923+0000","last_became_peered":"2026-02-25T15:01:38.791923+0000","last_unstale":"2026-02-25T15:07:49.598833+0000","last_undegraded":"2026-02-25T15:07:49.598833+0000","last_fullsized":"2026-02-25T15:07:49.598833+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:51:10.001377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062725172999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5],"acting":[11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"1.0","version":"24'32","reported_seq":680,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.555620+0000","last_change":"2026-02-25T15:01:30.868615+0000","last_active":"2026-02-25T15:07:49.555620+0000","last_peered":"2026-02-25T15:07:49.555620+0000","last_clean":"2026-02-25T15:07:49.555620+0000","last_became_active":"2026-02-25T15:01:30.851092+0000","last_became_peered":"2026-02-25T15:01:30.851092+0000","last_unstale":"2026-02-25T15:07:49.555620+0000","last_undegraded":"2026-02-25T15:07:49.555620+0000","last_fullsized":"2026-02-25T15:07:49.555620+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:29.405841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:29.405841+0000","last_clean_scrub_stamp":"2026-02-25T15:01:29.405841+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T02:31:46.315230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3],"acting":[8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.441645+0000","last_change":"2026-02-25T15:02:15.278232+0000","last_active":"2026-02-25T15:07:48.441645+0000","last_peered":"2026-02-25T15:07:48.441645+0000","last_clean":"2026-02-25T15:07:48.441645+0000","last_became_active":"2026-02-25T15:02:15.276113+0000","last_became_peered":"2026-02-25T15:02:15.276113+0000","last_unstale":"2026-02-25T15:07:48.441645+0000","last_undegraded":"2026-02-25T15:07:48.441645+0000","last_fullsized":"2026-02-25T15:07:48.441645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:15:46.080630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":619,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.434205+0000","last_change":"2026-02-25T15:02:14.248177+0000","last_active":"2026-02-25T15:07:48.434205+0000","last_peered":"2026-02-25T15:07:48.434205+0000","last_clean":"2026-02-25T15:07:48.434205+0000","last_became_active":"2026-02-25T15:02:14.243187+0000","last_became_peered":"2026-02-25T15:02:14.243187+0000","last_unstale":"2026-02-25T15:07:48.434205+0000","last_undegraded":"2026-02-25T15:07:48.434205+0000","last_fullsized":"2026-02-25T15:07:48.434205+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:34:38.222587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,10],"acting":[14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":619,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.436183+0000","last_change":"2026-02-25T15:02:13.817879+0000","last_active":"2026-02-25T15:07:48.436183+0000","last_peered":"2026-02-25T15:07:48.436183+0000","last_clean":"2026-02-25T15:07:48.436183+0000","last_became_active":"2026-02-25T15:02:13.815222+0000","last_became_peered":"2026-02-25T15:02:13.815222+0000","last_unstale":"2026-02-25T15:07:48.436183+0000","last_undegraded":"2026-02-25T15:07:48.436183+0000","last_fullsized":"2026-02-25T15:07:48.436183+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:31:52.870792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,9],"acting":[14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.567169+0000","last_change":"2026-02-25T15:02:15.251677+0000","last_active":"2026-02-25T15:07:50.567169+0000","last_peered":"2026-02-25T15:07:50.567169+0000","last_clean":"2026-02-25T15:07:50.567169+0000","last_became_active":"2026-02-25T15:02:15.249084+0000","last_became_peered":"2026-02-25T15:02:15.249084+0000","last_unstale":"2026-02-25T15:07:50.567169+0000","last_undegraded":"2026-02-25T15:07:50.567169+0000","last_fullsized":"2026-02-25T15:07:50.567169+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T02:14:00.305009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":621,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.563726+0000","last_change":"2026-02-25T15:02:14.838721+0000","last_active":"2026-02-25T15:07:50.563726+0000","last_peered":"2026-02-25T15:07:50.563726+0000","last_clean":"2026-02-25T15:07:50.563726+0000","last_became_active":"2026-02-25T15:02:14.836539+0000","last_became_peered":"2026-02-25T15:02:14.836539+0000","last_unstale":"2026-02-25T15:07:50.563726+0000","last_undegraded":"2026-02-25T15:07:50.563726+0000","last_fullsized":"2026-02-25T15:07:50.563726+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T02:18:20.966419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,13],"acting":[12,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":621,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.569052+0000","last_change":"2026-02-25T15:02:15.040439+0000","last_active":"2026-02-25T15:07:50.569052+0000","last_peered":"2026-02-25T15:07:50.569052+0000","last_clean":"2026-02-25T15:07:50.569052+0000","last_became_active":"2026-02-25T15:02:15.038261+0000","last_became_peered":"2026-02-25T15:02:15.038261+0000","last_unstale":"2026-02-25T15:07:50.569052+0000","last_undegraded":"2026-02-25T15:07:50.569052+0000","last_fullsized":"2026-02-25T15:07:50.569052+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:38:10.779195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,7],"acting":[12,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":619,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.561849+0000","last_change":"2026-02-25T15:02:13.433671+0000","last_active":"2026-02-25T15:07:49.561849+0000","last_peered":"2026-02-25T15:07:49.561849+0000","last_clean":"2026-02-25T15:07:49.561849+0000","last_became_active":"2026-02-25T15:02:13.426671+0000","last_became_peered":"2026-02-25T15:02:13.426671+0000","last_unstale":"2026-02-25T15:07:49.561849+0000","last_undegraded":"2026-02-25T15:07:49.561849+0000","last_fullsized":"2026-02-25T15:07:49.561849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:54:40.464012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10],"acting":[8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.571571+0000","last_change":"2026-02-25T15:02:14.942239+0000","last_active":"2026-02-25T15:07:50.571571+0000","last_peered":"2026-02-25T15:07:50.571571+0000","last_clean":"2026-02-25T15:07:50.571571+0000","last_became_active":"2026-02-25T15:02:14.934430+0000","last_became_peered":"2026-02-25T15:02:14.934430+0000","last_unstale":"2026-02-25T15:07:50.571571+0000","last_undegraded":"2026-02-25T15:07:50.571571+0000","last_fullsized":"2026-02-25T15:07:50.571571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:23:42.860020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8],"acting":[9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":621,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.565359+0000","last_change":"2026-02-25T15:02:14.166407+0000","last_active":"2026-02-25T15:07:50.565359+0000","last_peered":"2026-02-25T15:07:50.565359+0000","last_clean":"2026-02-25T15:07:50.565359+0000","last_became_active":"2026-02-25T15:02:14.164238+0000","last_became_peered":"2026-02-25T15:02:14.164238+0000","last_unstale":"2026-02-25T15:07:50.565359+0000","last_undegraded":"2026-02-25T15:07:50.565359+0000","last_fullsized":"2026-02-25T15:07:50.565359+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:06:11.961410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,15],"acting":[12,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.565158+0000","last_change":"2026-02-25T15:02:15.082986+0000","last_active":"2026-02-25T15:07:50.565158+0000","last_peered":"2026-02-25T15:07:50.565158+0000","last_clean":"2026-02-25T15:07:50.565158+0000","last_became_active":"2026-02-25T15:02:15.078177+0000","last_became_peered":"2026-02-25T15:02:15.078177+0000","last_unstale":"2026-02-25T15:07:50.565158+0000","last_undegraded":"2026-02-25T15:07:50.565158+0000","last_fullsized":"2026-02-25T15:07:50.565158+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:27:42.619819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10],"acting":[9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.563730+0000","last_change":"2026-02-25T15:02:14.239902+0000","last_active":"2026-02-25T15:07:50.563730+0000","last_peered":"2026-02-25T15:07:50.563730+0000","last_clean":"2026-02-25T15:07:50.563730+0000","last_became_active":"2026-02-25T15:02:14.237082+0000","last_became_peered":"2026-02-25T15:02:14.237082+0000","last_unstale":"2026-02-25T15:07:50.563730+0000","last_undegraded":"2026-02-25T15:07:50.563730+0000","last_fullsized":"2026-02-25T15:07:50.563730+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:57:36.384884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,14],"acting":[9,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.437951+0000","last_change":"2026-02-25T15:02:14.132167+0000","last_active":"2026-02-25T15:07:48.437951+0000","last_peered":"2026-02-25T15:07:48.437951+0000","last_clean":"2026-02-25T15:07:48.437951+0000","last_became_active":"2026-02-25T15:02:14.116657+0000","last_became_peered":"2026-02-25T15:02:14.116657+0000","last_unstale":"2026-02-25T15:07:48.437951+0000","last_undegraded":"2026-02-25T15:07:48.437951+0000","last_fullsized":"2026-02-25T15:07:48.437951+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:23:36.296106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":16}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280591,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33644,"kb_used_data":6684,"kb_used_omap":5740,"kb_used_meta":21203,"kb_avail":94338196,"statfs":{"total":96636764160,"available":96602312704,"internally_reserved":0,"allocated":6844416,"data_stored":4222855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5878530,"internal_metadata":21712126},"hb_peers":[0,1,3,4,5,6,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280591,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6676,"kb_used_omap":203,"kb_used_meta":26740,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6836224,"data_stored":4222831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":208491,"internal_metadata":27382165},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280590,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34088,"kb_used_data":7128,"kb_used_omap":203,"kb_used_meta":26740,"kb_avail":94337752,"statfs":{"total":96636764160,"available":96601858048,"internally_reserved":0,"allocated":7299072,"data_stored":4682111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":208377,"internal_metadata":27382279},"hb_peers":[1,2,4,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313295,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51668,"kb_used_data":6660,"kb_used_omap":7601,"kb_used_meta":37390,"kb_avail":94320172,"statfs":{"total":96636764160,"available":96583856128,"internally_reserved":0,"allocated":6819840,"data_stored":4212890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7784319,"internal_metadata":38287489},"hb_peers":[0,2,3,4,5,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":14,"up_from":21,"seq":90194313294,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33604,"kb_used_data":6644,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338236,"statfs":{"total":96636764160,"available":96602353664,"internally_reserved":0,"allocated":6803456,"data_stored":4200771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210523,"internal_metadata":27380133},"hb_peers":[0,1,2,3,4,5,6,8,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313294,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33620,"kb_used_data":6660,"kb_used_omap":203,"kb_used_meta":26740,"kb_avail":94338220,"statfs":{"total":96636764160,"available":96602337280,"internally_reserved":0,"allocated":6819840,"data_stored":4212890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":207913,"internal_metadata":27382743},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313294,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33620,"kb_used_data":6660,"kb_used_omap":204,"kb_used_meta":26739,"kb_avail":94338220,"statfs":{"total":96636764160,"available":96602337280,"internally_reserved":0,"allocated":6819840,"data_stored":4212890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":209288,"internal_metadata":27381368},"hb_peers":[1,2,3,4,5,6,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313294,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33620,"kb_used_data":6660,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338220,"statfs":{"total":96636764160,"available":96602337280,"internally_reserved":0,"allocated":6819840,"data_stored":4212890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":209978,"internal_metadata":27380678},"hb_peers":[1,3,5,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313294,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33620,"kb_used_data":6660,"kb_used_omap":204,"kb_used_meta":26739,"kb_avail":94338220,"statfs":{"total":96636764160,"available":96602337280,"internally_reserved":0,"allocated":6819840,"data_stored":4212890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":209789,"internal_metadata":27380867},"hb_peers":[0,1,2,3,4,5,6,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313295,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6676,"kb_used_omap":203,"kb_used_meta":26740,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6836224,"data_stored":4222831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":208876,"internal_metadata":27381780},"hb_peers":[1,3,5,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313294,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6676,"kb_used_omap":206,"kb_used_meta":26737,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6836224,"data_stored":4222831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":211548,"internal_metadata":27379108},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313295,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34088,"kb_used_data":7128,"kb_used_omap":206,"kb_used_meta":26737,"kb_avail":94337752,"statfs":{"total":96636764160,"available":96601858048,"internally_reserved":0,"allocated":7299072,"data_stored":4682111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":211268,"internal_metadata":27379388},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313294,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6676,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6836224,"data_stored":4222831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210167,"internal_metadata":27380489},"hb_peers":[0,2,4,5,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313294,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33624,"kb_used_data":6664,"kb_used_omap":5825,"kb_used_meta":21118,"kb_avail":94338216,"statfs":{"total":96636764160,"available":96602333184,"internally_reserved":0,"allocated":6823936,"data_stored":4212895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5965730,"internal_metadata":21624926},"hb_peers":[0,1,2,4,5,6,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313295,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33624,"kb_used_data":6664,"kb_used_omap":203,"kb_used_meta":26740,"kb_avail":94338216,"statfs":{"total":96636764160,"available":96602333184,"internally_reserved":0,"allocated":6823936,"data_stored":4212909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":208021,"internal_metadata":27382635},"hb_peers":[0,1,2,3,4,5,7,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313295,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6676,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6836224,"data_stored":4222831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210527,"internal_metadata":27380129},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T15:07:52.752 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-25T15:07:52.752 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T15:07:52.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.818+0000 7ff391ae8640 10 monclient: get_monmap_and_config 2026-02-25T15:07:52.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.818+0000 7ff391ae8640 10 monclient: build_initial_monmap 2026-02-25T15:07:52.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.818+0000 7ff391ae8640 10 monclient: monmap: 2026-02-25T15:07:52.821 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:52.821 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.822 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:52.820278+0000 2026-02-25T15:07:52.822 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:52.820278+0000 2026-02-25T15:07:52.822 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:52.822 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.822 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:52.822 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:52.822 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:52.822 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.819+0000 7ff391ae8640 1 Processor -- start 2026-02-25T15:07:52.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.819+0000 7ff391ae8640 1 -- start start 2026-02-25T15:07:52.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.819+0000 7ff391ae8640 10 monclient: init 2026-02-25T15:07:52.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.819+0000 7ff391ae8640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:52.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.819+0000 7ff391ae8640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:07:52.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.819+0000 7ff391ae8640 10 monclient(hunting): picked mon.noname-a con 0x7ff38c055bf0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:52.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.819+0000 7ff391ae8640 10 monclient(hunting): picked mon.noname-c con 0x7ff38c164040 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:52.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.819+0000 7ff391ae8640 10 monclient(hunting): picked mon.noname-b con 0x7ff38c055fc0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:52.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.819+0000 7ff391ae8640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff38c0b3a00 con 0x7ff38c055bf0 2026-02-25T15:07:52.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.819+0000 7ff391ae8640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff38c0b3c70 con 0x7ff38c055fc0 2026-02-25T15:07:52.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.819+0000 7ff391ae8640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff38c0b3f50 con 0x7ff38c164040 2026-02-25T15:07:52.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.819+0000 7ff391ae8640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:52.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.819+0000 7ff391ae8640 10 monclient(hunting): authenticate will time out at 3362.420903s 2026-02-25T15:07:52.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.820+0000 7ff38b7fe640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7ff38c055bf0 0x7ff38c0b1f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:32952/0 (socket says 10.20.193.90:32952) 2026-02-25T15:07:52.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.820+0000 7ff38b7fe640 1 -- 10.20.193.90:0/1913002674 learned_addr learned my addr 10.20.193.90:0/1913002674 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:52.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 1 -- 10.20.193.90:0/1913002674 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4065321821 0 0) 0x7ff38c0b3a00 con 0x7ff38c055bf0 2026-02-25T15:07:52.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 10 monclient(hunting): my global_id is 30771 2026-02-25T15:07:52.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 1 -- 10.20.193.90:0/1913002674 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff370003610 con 0x7ff38c055bf0 2026-02-25T15:07:52.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 1 -- 10.20.193.90:0/1913002674 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3049975042 0 0) 0x7ff38c0b3f50 con 0x7ff38c164040 2026-02-25T15:07:52.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 10 monclient(hunting): my global_id is 28541 2026-02-25T15:07:52.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 1 -- 10.20.193.90:0/1913002674 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff38c0b3a00 con 0x7ff38c164040 2026-02-25T15:07:52.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 1 -- 10.20.193.90:0/1913002674 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3843238265 0 0) 0x7ff38c0b3c70 con 0x7ff38c055fc0 2026-02-25T15:07:52.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 10 monclient(hunting): my global_id is 27403 2026-02-25T15:07:52.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.821+0000 7ff38a7fc640 1 -- 10.20.193.90:0/1913002674 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff38c0b3f50 con 0x7ff38c055fc0 2026-02-25T15:07:52.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff38a7fc640 1 -- 10.20.193.90:0/1913002674 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 362692058 0 0) 0x7ff370003610 con 0x7ff38c055bf0 2026-02-25T15:07:52.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff38a7fc640 1 -- 10.20.193.90:0/1913002674 >> v1:10.20.193.94:6789/0 conn(0x7ff38c164040 legacy=0x7ff38c056690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff38a7fc640 1 -- 10.20.193.90:0/1913002674 >> v1:10.20.193.93:6789/0 conn(0x7ff38c055fc0 legacy=0x7ff38c0b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff38a7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:07:52.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff38a7fc640 1 monclient: found mon.noname-a 2026-02-25T15:07:52.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff38a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:52.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff38a7fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:07:52.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff38a7fc640 1 -- 10.20.193.90:0/1913002674 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c0b4230 con 0x7ff38c055bf0 2026-02-25T15:07:52.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff38a7fc640 10 monclient: _finish_auth 0 2026-02-25T15:07:52.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff38a7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:07:52.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff38a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:52.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff38a7fc640 1 -- 10.20.193.90:0/1913002674 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff378002df0 con 0x7ff38c055bf0 2026-02-25T15:07:52.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff38a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:52.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff38a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:52.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff38a7fc640 10 monclient: dump: 2026-02-25T15:07:52.836 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.836 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.836 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.837 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.837 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.837 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.837 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.837 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.837 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.837 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff391ae8640 5 monclient: authenticate success, global_id 30771 2026-02-25T15:07:52.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.823+0000 7ff391ae8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:52.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff38a7fc640 1 -- 10.20.193.90:0/1913002674 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff378002fc0 con 0x7ff38c055bf0 2026-02-25T15:07:52.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff38a7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:52.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff38a7fc640 1 -- 10.20.193.90:0/1913002674 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff378003400 con 0x7ff38c055bf0 2026-02-25T15:07:52.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff38a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:52.838 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff38a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:52.838 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff38a7fc640 10 monclient: dump: 2026-02-25T15:07:52.838 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.838 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.838 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.838 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.838 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.838 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.838 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.838 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.838 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.838 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff391ae8640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:52.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff391ae8640 4 set_mon_vals no callback set 2026-02-25T15:07:52.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff391ae8640 10 monclient: shutdown 2026-02-25T15:07:52.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff391ae8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:52.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff391ae8640 1 -- 10.20.193.90:0/1913002674 >> v1:10.20.193.90:6789/0 conn(0x7ff38c055bf0 legacy=0x7ff38c0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff391ae8640 1 -- 10.20.193.90:0/1913002674 shutdown_connections 2026-02-25T15:07:52.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff391ae8640 1 -- 10.20.193.90:0/1913002674 >> 10.20.193.90:0/1913002674 conn(0x7ff38c0b7bb0 msgr2=0x7ff38c0b7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:52.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff391ae8640 1 -- 10.20.193.90:0/1913002674 shutdown_connections 2026-02-25T15:07:52.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff391ae8640 1 -- 10.20.193.90:0/1913002674 wait complete. 2026-02-25T15:07:52.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff391ae8640 10 monclient: build_initial_monmap 2026-02-25T15:07:52.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.824+0000 7ff391ae8640 10 monclient: monmap: 2026-02-25T15:07:52.839 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:52.839 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:52.840 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:52.840 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:52.840 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:52.840 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:52.840 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:52.840 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:52.840 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:52.840 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 1 librados: starting msgr at 2026-02-25T15:07:52.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 1 librados: starting objecter 2026-02-25T15:07:52.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:52.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 1 Processor -- start 2026-02-25T15:07:52.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 1 -- start start 2026-02-25T15:07:52.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 1 librados: setting wanted keys 2026-02-25T15:07:52.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 1 librados: calling monclient init 2026-02-25T15:07:52.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 10 monclient: init 2026-02-25T15:07:52.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:52.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:07:52.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 10 monclient(hunting): picked mon.noname-a con 0x7ff38c055bf0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:52.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 10 monclient(hunting): picked mon.noname-c con 0x7ff38c055fc0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:52.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 10 monclient(hunting): picked mon.noname-b con 0x7ff38c164040 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:52.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff38c154650 con 0x7ff38c055bf0 2026-02-25T15:07:52.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff38c139d40 con 0x7ff38c164040 2026-02-25T15:07:52.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff38c13a020 con 0x7ff38c055fc0 2026-02-25T15:07:52.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:52.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff391ae8640 10 monclient(hunting): authenticate will time out at 3362.426652s 2026-02-25T15:07:52.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff38b7fe640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7ff38c055bf0 0x7ff38c153f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:32954/0 (socket says 10.20.193.90:32954) 2026-02-25T15:07:52.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.825+0000 7ff38b7fe640 1 -- 10.20.193.90:0/2275692501 learned_addr learned my addr 10.20.193.90:0/2275692501 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:52.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.826+0000 7ff367fff640 10 client.?.objecter ms_handle_connect 0x7ff38c055bf0 2026-02-25T15:07:52.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.826+0000 7ff367fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:52.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.826+0000 7ff367fff640 10 client.?.objecter ms_handle_connect 0x7ff38c164040 2026-02-25T15:07:52.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.826+0000 7ff367fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:52.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.826+0000 7ff367fff640 10 client.?.objecter ms_handle_connect 0x7ff38c055fc0 2026-02-25T15:07:52.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.826+0000 7ff367fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:52.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.826+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4145178997 0 0) 0x7ff38c154650 con 0x7ff38c055bf0 2026-02-25T15:07:52.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 10 monclient(hunting): my global_id is 30774 2026-02-25T15:07:52.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff35c003610 con 0x7ff38c055bf0 2026-02-25T15:07:52.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3372279796 0 0) 0x7ff38c139d40 con 0x7ff38c164040 2026-02-25T15:07:52.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 10 monclient(hunting): my global_id is 27406 2026-02-25T15:07:52.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff38c154650 con 0x7ff38c164040 2026-02-25T15:07:52.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3540123272 0 0) 0x7ff38c13a020 con 0x7ff38c055fc0 2026-02-25T15:07:52.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:52.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:52.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:52.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 10 monclient(hunting): my global_id is 28544 2026-02-25T15:07:52.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.827+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff38c139d40 con 0x7ff38c055fc0 2026-02-25T15:07:52.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1634277965 0 0) 0x7ff38c139d40 con 0x7ff38c055fc0 2026-02-25T15:07:52.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff38c13a020 con 0x7ff38c055fc0 2026-02-25T15:07:52.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff37c002ff0 con 0x7ff38c055fc0 2026-02-25T15:07:52.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:52.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:07:52.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 10 monclient(hunting): dump: 2026-02-25T15:07:52.844 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.844 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.844 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.845 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.845 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.845 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.845 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.845 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.845 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.845 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:52.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3364971436 0 0) 0x7ff38c154650 con 0x7ff38c164040 2026-02-25T15:07:52.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff38c139d40 con 0x7ff38c164040 2026-02-25T15:07:52.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff380003410 con 0x7ff38c164040 2026-02-25T15:07:52.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:52.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:52.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 10 monclient(hunting): dump: 2026-02-25T15:07:52.846 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.846 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.846 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.846 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.846 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.846 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.846 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.846 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.846 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.846 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3563614397 0 0) 0x7ff35c003610 con 0x7ff38c055bf0 2026-02-25T15:07:52.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff38c154650 con 0x7ff38c055bf0 2026-02-25T15:07:52.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff378003310 con 0x7ff38c055bf0 2026-02-25T15:07:52.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:52.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:52.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.829+0000 7ff367fff640 10 monclient(hunting): dump: 2026-02-25T15:07:52.847 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.847 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.847 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.847 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.847 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.847 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.847 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.847 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.848 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.848 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4010161886 0 0) 0x7ff38c13a020 con 0x7ff38c055fc0 2026-02-25T15:07:52.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 >> v1:10.20.193.93:6789/0 conn(0x7ff38c164040 legacy=0x7ff38c14c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 >> v1:10.20.193.90:6789/0 conn(0x7ff38c055bf0 legacy=0x7ff38c153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:52.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff367fff640 10 monclient: _finish_hunting 0 2026-02-25T15:07:52.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff367fff640 1 monclient: found mon.c 2026-02-25T15:07:52.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff367fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:52.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff367fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:52.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c13a300 con 0x7ff38c055fc0 2026-02-25T15:07:52.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff367fff640 10 monclient: _finish_auth 0 2026-02-25T15:07:52.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff367fff640 10 monclient: _check_auth_tickets 2026-02-25T15:07:52.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff367fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:52.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff391ae8640 5 monclient: authenticate success, global_id 28544 2026-02-25T15:07:52.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff391ae8640 10 monclient: _renew_subs 2026-02-25T15:07:52.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff391ae8640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:52.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff391ae8640 1 -- 10.20.193.90:0/2275692501 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff38c13a5e0 con 0x7ff38c055fc0 2026-02-25T15:07:52.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff391ae8640 10 client.28544.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:52.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff391ae8640 10 monclient: _renew_subs 2026-02-25T15:07:52.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff391ae8640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:52.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff391ae8640 1 -- 10.20.193.90:0/2275692501 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff38c139930 con 0x7ff38c055fc0 2026-02-25T15:07:52.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff391ae8640 1 librados: init done 2026-02-25T15:07:52.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff391ae8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:52.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff391ae8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:52.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff391ae8640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:52.850 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.831+0000 7ff391ae8640 1 -- 10.20.193.90:0/2275692501 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff38c0b5150 con 0x7ff38c055fc0 2026-02-25T15:07:52.851 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.839+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff37c003470 con 0x7ff38c055fc0 2026-02-25T15:07:52.851 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.839+0000 7ff367fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:52.851 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.839+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff37c003880 con 0x7ff38c055fc0 2026-02-25T15:07:52.851 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.839+0000 7ff367fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:52.851 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.839+0000 7ff367fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:52.851 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.839+0000 7ff389ffb640 4 set_mon_vals no callback set 2026-02-25T15:07:52.851 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.839+0000 7ff367fff640 10 monclient: dump: 2026-02-25T15:07:52.852 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:52.852 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:52.852 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.852 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:52.852 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:52.852 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:52.852 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:52.852 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:52.852 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:52.852 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:52.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.840+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7ff37c003a60 con 0x7ff38c055fc0 2026-02-25T15:07:52.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.840+0000 7ff38b7fe640 10 monclient: get_auth_request con 0x7ff35c07a040 auth_method 0 2026-02-25T15:07:52.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.841+0000 7ff367fff640 10 client.28544.objecter ms_handle_connect 0x7ff35c07a040 2026-02-25T15:07:52.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.841+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(337..337 src has 1..337) ==== 8959+0+0 (unknown 3556021173 0 0) 0x7ff37c0947a0 con 0x7ff38c055fc0 2026-02-25T15:07:52.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.841+0000 7ff367fff640 10 client.28544.objecter ms_dispatch2 0x7ff38c000b90 osd_map(337..337 src has 1..337) 2026-02-25T15:07:52.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.841+0000 7ff367fff640 3 client.28544.objecter handle_osd_map got epochs [337,337] > 0 2026-02-25T15:07:52.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.841+0000 7ff367fff640 3 client.28544.objecter handle_osd_map decoding full epoch 337 2026-02-25T15:07:52.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.841+0000 7ff367fff640 20 client.28544.objecter dump_active .. 0 homeless 2026-02-25T15:07:52.928 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.927+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7ff38c0b5150 con 0x7ff38c055fc0 2026-02-25T15:07:52.929 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.927+0000 7ff367fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:52.929 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:52.927+0000 7ff367fff640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:53.033 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.030+0000 7ff391ae8640 1 -- 10.20.193.90:0/2275692501 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff38c13a800 con 0x7ff35c07a040 2026-02-25T15:07:53.035 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.032+0000 7ff367fff640 1 -- 10.20.193.90:0/2275692501 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+125395 (unknown 2965378022 0 1250104598) 0x7ff38c13a800 con 0x7ff35c07a040 2026-02-25T15:07:53.036 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:07:53.037 INFO:teuthology.orchestra.run.trial090.stderr:dumped all 2026-02-25T15:07:53.038 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.033+0000 7ff391ae8640 10 librados: watch_flush enter 2026-02-25T15:07:53.038 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.033+0000 7ff391ae8640 10 librados: watch_flush exit 2026-02-25T15:07:53.038 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.033+0000 7ff391ae8640 20 client.28544.objecter shutdown clearing up homeless session... 2026-02-25T15:07:53.038 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.033+0000 7ff391ae8640 10 client.28544.objecter successfully canceled tick 2026-02-25T15:07:53.039 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.033+0000 7ff391ae8640 1 -- 10.20.193.90:0/2275692501 >> v1:10.20.193.189:6800/3647375982 conn(0x7ff35c07a040 legacy=0x7ff35c09a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:53.039 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.033+0000 7ff391ae8640 10 monclient: shutdown 2026-02-25T15:07:53.039 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.033+0000 7ff391ae8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:53.039 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.033+0000 7ff391ae8640 1 -- 10.20.193.90:0/2275692501 >> v1:10.20.193.94:6789/0 conn(0x7ff38c055fc0 legacy=0x7ff38c15e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:53.039 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.033+0000 7ff391ae8640 1 -- 10.20.193.90:0/2275692501 shutdown_connections 2026-02-25T15:07:53.039 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.033+0000 7ff391ae8640 1 -- 10.20.193.90:0/2275692501 >> 10.20.193.90:0/2275692501 conn(0x7ff38c0b7bb0 msgr2=0x7ff38c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:53.039 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.033+0000 7ff391ae8640 1 -- 10.20.193.90:0/2275692501 shutdown_connections 2026-02-25T15:07:53.039 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.033+0000 7ff391ae8640 1 -- 10.20.193.90:0/2275692501 wait complete. 2026-02-25T15:07:53.039 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.033+0000 7ff391ae8640 1 librados: shutdown 2026-02-25T15:07:53.044 INFO:teuthology.orchestra.run.trial090.stdout:{"pg_ready":true,"pg_map":{"version":530,"stamp":"2026-02-25T15:07:51.593123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":56,"num_read_kb":47,"num_write":70,"num_write_kb":597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":557000,"kb_used_data":107592,"kb_used_omap":21829,"kb_used_meta":427322,"kb_avail":1509392440,"statfs":{"total":1546188226560,"available":1545617858560,"internally_reserved":0,"allocated":110174208,"data_stored":68392257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22353345,"internal_metadata":437578303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":78,"apply_latency_ms":78,"commit_latency_ns":78000000,"apply_latency_ns":78000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.158859"},"pg_stats":[{"pgid":"25.5","version":"0'0","reported_seq":619,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.437533+0000","last_change":"2026-02-25T15:02:14.833079+0000","last_active":"2026-02-25T15:07:48.437533+0000","last_peered":"2026-02-25T15:07:48.437533+0000","last_clean":"2026-02-25T15:07:48.437533+0000","last_became_active":"2026-02-25T15:02:14.830708+0000","last_became_peered":"2026-02-25T15:02:14.830708+0000","last_unstale":"2026-02-25T15:07:48.437533+0000","last_undegraded":"2026-02-25T15:07:48.437533+0000","last_fullsized":"2026-02-25T15:07:48.437533+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:21:07.436552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,8],"acting":[14,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.569659+0000","last_change":"2026-02-25T15:02:14.703677+0000","last_active":"2026-02-25T15:07:50.569659+0000","last_peered":"2026-02-25T15:07:50.569659+0000","last_clean":"2026-02-25T15:07:50.569659+0000","last_became_active":"2026-02-25T15:02:14.686130+0000","last_became_peered":"2026-02-25T15:02:14.686130+0000","last_unstale":"2026-02-25T15:07:50.569659+0000","last_undegraded":"2026-02-25T15:07:50.569659+0000","last_fullsized":"2026-02-25T15:07:50.569659+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:37:58.016312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7],"acting":[9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.444074+0000","last_change":"2026-02-25T15:02:14.184346+0000","last_active":"2026-02-25T15:07:48.444074+0000","last_peered":"2026-02-25T15:07:48.444074+0000","last_clean":"2026-02-25T15:07:48.444074+0000","last_became_active":"2026-02-25T15:02:14.182253+0000","last_became_peered":"2026-02-25T15:02:14.182253+0000","last_unstale":"2026-02-25T15:07:48.444074+0000","last_undegraded":"2026-02-25T15:07:48.444074+0000","last_fullsized":"2026-02-25T15:07:48.444074+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:10:33.826830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.442617+0000","last_change":"2026-02-25T15:02:14.262914+0000","last_active":"2026-02-25T15:07:48.442617+0000","last_peered":"2026-02-25T15:07:48.442617+0000","last_clean":"2026-02-25T15:07:48.442617+0000","last_became_active":"2026-02-25T15:02:14.258456+0000","last_became_peered":"2026-02-25T15:02:14.258456+0000","last_unstale":"2026-02-25T15:07:48.442617+0000","last_undegraded":"2026-02-25T15:07:48.442617+0000","last_fullsized":"2026-02-25T15:07:48.442617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:29:58.464087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"33'11","reported_seq":645,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.505217+0000","last_change":"2026-02-25T15:02:15.833921+0000","last_active":"2026-02-25T15:07:49.505217+0000","last_peered":"2026-02-25T15:07:49.505217+0000","last_clean":"2026-02-25T15:07:49.505217+0000","last_became_active":"2026-02-25T15:02:15.819620+0000","last_became_peered":"2026-02-25T15:02:15.819620+0000","last_unstale":"2026-02-25T15:07:49.505217+0000","last_undegraded":"2026-02-25T15:07:49.505217+0000","last_fullsized":"2026-02-25T15:07:49.505217+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:40:10.738663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,13],"acting":[10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":616,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.562854+0000","last_change":"2026-02-25T15:02:13.775077+0000","last_active":"2026-02-25T15:07:49.562854+0000","last_peered":"2026-02-25T15:07:49.562854+0000","last_clean":"2026-02-25T15:07:49.562854+0000","last_became_active":"2026-02-25T15:02:13.773022+0000","last_became_peered":"2026-02-25T15:02:13.773022+0000","last_unstale":"2026-02-25T15:07:49.562854+0000","last_undegraded":"2026-02-25T15:07:49.562854+0000","last_fullsized":"2026-02-25T15:07:49.562854+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:04:02.164409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":620,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.558382+0000","last_change":"2026-02-25T15:02:14.036683+0000","last_active":"2026-02-25T15:07:49.558382+0000","last_peered":"2026-02-25T15:07:49.558382+0000","last_clean":"2026-02-25T15:07:49.558382+0000","last_became_active":"2026-02-25T15:02:14.028952+0000","last_became_peered":"2026-02-25T15:02:14.028952+0000","last_unstale":"2026-02-25T15:07:49.558382+0000","last_undegraded":"2026-02-25T15:07:49.558382+0000","last_fullsized":"2026-02-25T15:07:49.558382+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:24:37.223453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,13],"acting":[5,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":619,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.584196+0000","last_change":"2026-02-25T15:02:14.938617+0000","last_active":"2026-02-25T15:07:50.584196+0000","last_peered":"2026-02-25T15:07:50.584196+0000","last_clean":"2026-02-25T15:07:50.584196+0000","last_became_active":"2026-02-25T15:02:14.936242+0000","last_became_peered":"2026-02-25T15:02:14.936242+0000","last_unstale":"2026-02-25T15:07:50.584196+0000","last_undegraded":"2026-02-25T15:07:50.584196+0000","last_fullsized":"2026-02-25T15:07:50.584196+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:59:06.072509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.579130+0000","last_change":"2026-02-25T15:02:13.982149+0000","last_active":"2026-02-25T15:07:48.579130+0000","last_peered":"2026-02-25T15:07:48.579130+0000","last_clean":"2026-02-25T15:07:48.579130+0000","last_became_active":"2026-02-25T15:02:13.976916+0000","last_became_peered":"2026-02-25T15:02:13.976916+0000","last_unstale":"2026-02-25T15:07:48.579130+0000","last_undegraded":"2026-02-25T15:07:48.579130+0000","last_fullsized":"2026-02-25T15:07:48.579130+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:19:44.345322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":616,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.565247+0000","last_change":"2026-02-25T15:02:13.384676+0000","last_active":"2026-02-25T15:07:49.565247+0000","last_peered":"2026-02-25T15:07:49.565247+0000","last_clean":"2026-02-25T15:07:49.565247+0000","last_became_active":"2026-02-25T15:02:13.382450+0000","last_became_peered":"2026-02-25T15:02:13.382450+0000","last_unstale":"2026-02-25T15:07:49.565247+0000","last_undegraded":"2026-02-25T15:07:49.565247+0000","last_fullsized":"2026-02-25T15:07:49.565247+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:23:31.158323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10],"acting":[0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":617,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.605481+0000","last_change":"2026-02-25T15:02:13.940723+0000","last_active":"2026-02-25T15:07:49.605481+0000","last_peered":"2026-02-25T15:07:49.605481+0000","last_clean":"2026-02-25T15:07:49.605481+0000","last_became_active":"2026-02-25T15:02:13.934323+0000","last_became_peered":"2026-02-25T15:02:13.934323+0000","last_unstale":"2026-02-25T15:07:49.605481+0000","last_undegraded":"2026-02-25T15:07:49.605481+0000","last_fullsized":"2026-02-25T15:07:49.605481+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:14:02.898606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10],"acting":[11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":620,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.567595+0000","last_change":"2026-02-25T15:02:14.071335+0000","last_active":"2026-02-25T15:07:49.567595+0000","last_peered":"2026-02-25T15:07:49.567595+0000","last_clean":"2026-02-25T15:07:49.567595+0000","last_became_active":"2026-02-25T15:02:14.063362+0000","last_became_peered":"2026-02-25T15:02:14.063362+0000","last_unstale":"2026-02-25T15:07:49.567595+0000","last_undegraded":"2026-02-25T15:07:49.567595+0000","last_fullsized":"2026-02-25T15:07:49.567595+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:41:34.565838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,14],"acting":[5,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.440309+0000","last_change":"2026-02-25T15:02:13.851333+0000","last_active":"2026-02-25T15:07:48.440309+0000","last_peered":"2026-02-25T15:07:48.440309+0000","last_clean":"2026-02-25T15:07:48.440309+0000","last_became_active":"2026-02-25T15:02:13.849038+0000","last_became_peered":"2026-02-25T15:02:13.849038+0000","last_unstale":"2026-02-25T15:07:48.440309+0000","last_undegraded":"2026-02-25T15:07:48.440309+0000","last_fullsized":"2026-02-25T15:07:48.440309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:04:11.634095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":632,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.604755+0000","last_change":"2026-02-25T15:01:40.617020+0000","last_active":"2026-02-25T15:07:49.604755+0000","last_peered":"2026-02-25T15:07:49.604755+0000","last_clean":"2026-02-25T15:07:49.604755+0000","last_became_active":"2026-02-25T15:01:38.997857+0000","last_became_peered":"2026-02-25T15:01:38.997857+0000","last_unstale":"2026-02-25T15:07:49.604755+0000","last_undegraded":"2026-02-25T15:07:49.604755+0000","last_fullsized":"2026-02-25T15:07:49.604755+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:49:56.506302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062975700999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14],"acting":[11,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":617,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.546518+0000","last_change":"2026-02-25T15:02:13.715917+0000","last_active":"2026-02-25T15:07:49.546518+0000","last_peered":"2026-02-25T15:07:49.546518+0000","last_clean":"2026-02-25T15:07:49.546518+0000","last_became_active":"2026-02-25T15:02:13.713599+0000","last_became_peered":"2026-02-25T15:02:13.713599+0000","last_unstale":"2026-02-25T15:07:49.546518+0000","last_undegraded":"2026-02-25T15:07:49.546518+0000","last_fullsized":"2026-02-25T15:07:49.546518+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:15:39.798144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":636,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.567836+0000","last_change":"2026-02-25T15:01:41.128386+0000","last_active":"2026-02-25T15:07:50.567836+0000","last_peered":"2026-02-25T15:07:50.567836+0000","last_clean":"2026-02-25T15:07:50.567836+0000","last_became_active":"2026-02-25T15:01:38.782341+0000","last_became_peered":"2026-02-25T15:01:38.782341+0000","last_unstale":"2026-02-25T15:07:50.567836+0000","last_undegraded":"2026-02-25T15:07:50.567836+0000","last_fullsized":"2026-02-25T15:07:50.567836+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:07:32.765627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031008292,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,3],"acting":[12,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":619,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.563927+0000","last_change":"2026-02-25T15:02:13.858401+0000","last_active":"2026-02-25T15:07:50.563927+0000","last_peered":"2026-02-25T15:07:50.563927+0000","last_clean":"2026-02-25T15:07:50.563927+0000","last_became_active":"2026-02-25T15:02:13.846214+0000","last_became_peered":"2026-02-25T15:02:13.846214+0000","last_unstale":"2026-02-25T15:07:50.563927+0000","last_undegraded":"2026-02-25T15:07:50.563927+0000","last_fullsized":"2026-02-25T15:07:50.563927+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:30:33.308451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11],"acting":[3,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":634,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.596549+0000","last_change":"2026-02-25T15:01:41.143783+0000","last_active":"2026-02-25T15:07:50.596549+0000","last_peered":"2026-02-25T15:07:50.596549+0000","last_clean":"2026-02-25T15:07:50.596549+0000","last_became_active":"2026-02-25T15:01:38.731569+0000","last_became_peered":"2026-02-25T15:01:38.731569+0000","last_unstale":"2026-02-25T15:07:50.596549+0000","last_undegraded":"2026-02-25T15:07:50.596549+0000","last_fullsized":"2026-02-25T15:07:50.596549+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:00:25.099174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037093318,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15],"acting":[7,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":619,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.441335+0000","last_change":"2026-02-25T15:02:14.257499+0000","last_active":"2026-02-25T15:07:48.441335+0000","last_peered":"2026-02-25T15:07:48.441335+0000","last_clean":"2026-02-25T15:07:48.441335+0000","last_became_active":"2026-02-25T15:02:14.254843+0000","last_became_peered":"2026-02-25T15:02:14.254843+0000","last_unstale":"2026-02-25T15:07:48.441335+0000","last_undegraded":"2026-02-25T15:07:48.441335+0000","last_fullsized":"2026-02-25T15:07:48.441335+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:07:11.424289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2],"acting":[14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":634,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.593113+0000","last_change":"2026-02-25T15:01:41.139794+0000","last_active":"2026-02-25T15:07:50.593113+0000","last_peered":"2026-02-25T15:07:50.593113+0000","last_clean":"2026-02-25T15:07:50.593113+0000","last_became_active":"2026-02-25T15:01:38.780556+0000","last_became_peered":"2026-02-25T15:01:38.780556+0000","last_unstale":"2026-02-25T15:07:50.593113+0000","last_undegraded":"2026-02-25T15:07:50.593113+0000","last_fullsized":"2026-02-25T15:07:50.593113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:57:49.842137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041710525999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":616,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.560799+0000","last_change":"2026-02-25T15:02:14.919090+0000","last_active":"2026-02-25T15:07:49.560799+0000","last_peered":"2026-02-25T15:07:49.560799+0000","last_clean":"2026-02-25T15:07:49.560799+0000","last_became_active":"2026-02-25T15:02:14.894984+0000","last_became_peered":"2026-02-25T15:02:14.894984+0000","last_unstale":"2026-02-25T15:07:49.560799+0000","last_undegraded":"2026-02-25T15:07:49.560799+0000","last_fullsized":"2026-02-25T15:07:49.560799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:10:40.021124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":634,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.596471+0000","last_change":"2026-02-25T15:01:40.608781+0000","last_active":"2026-02-25T15:07:49.596471+0000","last_peered":"2026-02-25T15:07:49.596471+0000","last_clean":"2026-02-25T15:07:49.596471+0000","last_became_active":"2026-02-25T15:01:38.743941+0000","last_became_peered":"2026-02-25T15:01:38.743941+0000","last_unstale":"2026-02-25T15:07:49.596471+0000","last_undegraded":"2026-02-25T15:07:49.596471+0000","last_fullsized":"2026-02-25T15:07:49.596471+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:53:32.920667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.061484652000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13],"acting":[11,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":620,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.568600+0000","last_change":"2026-02-25T15:02:14.369913+0000","last_active":"2026-02-25T15:07:49.568600+0000","last_peered":"2026-02-25T15:07:49.568600+0000","last_clean":"2026-02-25T15:07:49.568600+0000","last_became_active":"2026-02-25T15:02:14.364385+0000","last_became_peered":"2026-02-25T15:02:14.364385+0000","last_unstale":"2026-02-25T15:07:49.568600+0000","last_undegraded":"2026-02-25T15:07:49.568600+0000","last_fullsized":"2026-02-25T15:07:49.568600+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:35:03.754835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":632,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.543792+0000","last_change":"2026-02-25T15:01:41.129594+0000","last_active":"2026-02-25T15:07:49.543792+0000","last_peered":"2026-02-25T15:07:49.543792+0000","last_clean":"2026-02-25T15:07:49.543792+0000","last_became_active":"2026-02-25T15:01:38.861789+0000","last_became_peered":"2026-02-25T15:01:38.861789+0000","last_unstale":"2026-02-25T15:07:49.543792+0000","last_undegraded":"2026-02-25T15:07:49.543792+0000","last_fullsized":"2026-02-25T15:07:49.543792+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:36:46.424677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032448785000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":617,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.598089+0000","last_change":"2026-02-25T15:02:15.000767+0000","last_active":"2026-02-25T15:07:49.598089+0000","last_peered":"2026-02-25T15:07:49.598089+0000","last_clean":"2026-02-25T15:07:49.598089+0000","last_became_active":"2026-02-25T15:02:14.998340+0000","last_became_peered":"2026-02-25T15:02:14.998340+0000","last_unstale":"2026-02-25T15:07:49.598089+0000","last_undegraded":"2026-02-25T15:07:49.598089+0000","last_fullsized":"2026-02-25T15:07:49.598089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:06:28.044352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5],"acting":[11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":632,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.562031+0000","last_change":"2026-02-25T15:01:41.116971+0000","last_active":"2026-02-25T15:07:49.562031+0000","last_peered":"2026-02-25T15:07:49.562031+0000","last_clean":"2026-02-25T15:07:49.562031+0000","last_became_active":"2026-02-25T15:01:38.737942+0000","last_became_peered":"2026-02-25T15:01:38.737942+0000","last_unstale":"2026-02-25T15:07:49.562031+0000","last_undegraded":"2026-02-25T15:07:49.562031+0000","last_fullsized":"2026-02-25T15:07:49.562031+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:51:33.138764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031295004000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13],"acting":[0,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":619,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.570012+0000","last_change":"2026-02-25T15:02:13.590713+0000","last_active":"2026-02-25T15:07:50.570012+0000","last_peered":"2026-02-25T15:07:50.570012+0000","last_clean":"2026-02-25T15:07:50.570012+0000","last_became_active":"2026-02-25T15:02:13.588299+0000","last_became_peered":"2026-02-25T15:02:13.588299+0000","last_unstale":"2026-02-25T15:07:50.570012+0000","last_undegraded":"2026-02-25T15:07:50.570012+0000","last_fullsized":"2026-02-25T15:07:50.570012+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:10:21.612478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":633,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.598833+0000","last_change":"2026-02-25T15:01:40.614625+0000","last_active":"2026-02-25T15:07:49.598833+0000","last_peered":"2026-02-25T15:07:49.598833+0000","last_clean":"2026-02-25T15:07:49.598833+0000","last_became_active":"2026-02-25T15:01:38.791923+0000","last_became_peered":"2026-02-25T15:01:38.791923+0000","last_unstale":"2026-02-25T15:07:49.598833+0000","last_undegraded":"2026-02-25T15:07:49.598833+0000","last_fullsized":"2026-02-25T15:07:49.598833+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:51:10.001377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062725172999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5],"acting":[11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"1.0","version":"24'32","reported_seq":680,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.555620+0000","last_change":"2026-02-25T15:01:30.868615+0000","last_active":"2026-02-25T15:07:49.555620+0000","last_peered":"2026-02-25T15:07:49.555620+0000","last_clean":"2026-02-25T15:07:49.555620+0000","last_became_active":"2026-02-25T15:01:30.851092+0000","last_became_peered":"2026-02-25T15:01:30.851092+0000","last_unstale":"2026-02-25T15:07:49.555620+0000","last_undegraded":"2026-02-25T15:07:49.555620+0000","last_fullsized":"2026-02-25T15:07:49.555620+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:29.405841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:29.405841+0000","last_clean_scrub_stamp":"2026-02-25T15:01:29.405841+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T02:31:46.315230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3],"acting":[8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.441645+0000","last_change":"2026-02-25T15:02:15.278232+0000","last_active":"2026-02-25T15:07:48.441645+0000","last_peered":"2026-02-25T15:07:48.441645+0000","last_clean":"2026-02-25T15:07:48.441645+0000","last_became_active":"2026-02-25T15:02:15.276113+0000","last_became_peered":"2026-02-25T15:02:15.276113+0000","last_unstale":"2026-02-25T15:07:48.441645+0000","last_undegraded":"2026-02-25T15:07:48.441645+0000","last_fullsized":"2026-02-25T15:07:48.441645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:15:46.080630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":619,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.434205+0000","last_change":"2026-02-25T15:02:14.248177+0000","last_active":"2026-02-25T15:07:48.434205+0000","last_peered":"2026-02-25T15:07:48.434205+0000","last_clean":"2026-02-25T15:07:48.434205+0000","last_became_active":"2026-02-25T15:02:14.243187+0000","last_became_peered":"2026-02-25T15:02:14.243187+0000","last_unstale":"2026-02-25T15:07:48.434205+0000","last_undegraded":"2026-02-25T15:07:48.434205+0000","last_fullsized":"2026-02-25T15:07:48.434205+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:34:38.222587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,10],"acting":[14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":619,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.436183+0000","last_change":"2026-02-25T15:02:13.817879+0000","last_active":"2026-02-25T15:07:48.436183+0000","last_peered":"2026-02-25T15:07:48.436183+0000","last_clean":"2026-02-25T15:07:48.436183+0000","last_became_active":"2026-02-25T15:02:13.815222+0000","last_became_peered":"2026-02-25T15:02:13.815222+0000","last_unstale":"2026-02-25T15:07:48.436183+0000","last_undegraded":"2026-02-25T15:07:48.436183+0000","last_fullsized":"2026-02-25T15:07:48.436183+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:31:52.870792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,9],"acting":[14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.567169+0000","last_change":"2026-02-25T15:02:15.251677+0000","last_active":"2026-02-25T15:07:50.567169+0000","last_peered":"2026-02-25T15:07:50.567169+0000","last_clean":"2026-02-25T15:07:50.567169+0000","last_became_active":"2026-02-25T15:02:15.249084+0000","last_became_peered":"2026-02-25T15:02:15.249084+0000","last_unstale":"2026-02-25T15:07:50.567169+0000","last_undegraded":"2026-02-25T15:07:50.567169+0000","last_fullsized":"2026-02-25T15:07:50.567169+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T02:14:00.305009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":621,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.563726+0000","last_change":"2026-02-25T15:02:14.838721+0000","last_active":"2026-02-25T15:07:50.563726+0000","last_peered":"2026-02-25T15:07:50.563726+0000","last_clean":"2026-02-25T15:07:50.563726+0000","last_became_active":"2026-02-25T15:02:14.836539+0000","last_became_peered":"2026-02-25T15:02:14.836539+0000","last_unstale":"2026-02-25T15:07:50.563726+0000","last_undegraded":"2026-02-25T15:07:50.563726+0000","last_fullsized":"2026-02-25T15:07:50.563726+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T02:18:20.966419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,13],"acting":[12,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":621,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.569052+0000","last_change":"2026-02-25T15:02:15.040439+0000","last_active":"2026-02-25T15:07:50.569052+0000","last_peered":"2026-02-25T15:07:50.569052+0000","last_clean":"2026-02-25T15:07:50.569052+0000","last_became_active":"2026-02-25T15:02:15.038261+0000","last_became_peered":"2026-02-25T15:02:15.038261+0000","last_unstale":"2026-02-25T15:07:50.569052+0000","last_undegraded":"2026-02-25T15:07:50.569052+0000","last_fullsized":"2026-02-25T15:07:50.569052+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:38:10.779195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,7],"acting":[12,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":619,"reported_epoch":335,"state":"active+clean","last_fresh":"2026-02-25T15:07:49.561849+0000","last_change":"2026-02-25T15:02:13.433671+0000","last_active":"2026-02-25T15:07:49.561849+0000","last_peered":"2026-02-25T15:07:49.561849+0000","last_clean":"2026-02-25T15:07:49.561849+0000","last_became_active":"2026-02-25T15:02:13.426671+0000","last_became_peered":"2026-02-25T15:02:13.426671+0000","last_unstale":"2026-02-25T15:07:49.561849+0000","last_undegraded":"2026-02-25T15:07:49.561849+0000","last_fullsized":"2026-02-25T15:07:49.561849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:54:40.464012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10],"acting":[8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.571571+0000","last_change":"2026-02-25T15:02:14.942239+0000","last_active":"2026-02-25T15:07:50.571571+0000","last_peered":"2026-02-25T15:07:50.571571+0000","last_clean":"2026-02-25T15:07:50.571571+0000","last_became_active":"2026-02-25T15:02:14.934430+0000","last_became_peered":"2026-02-25T15:02:14.934430+0000","last_unstale":"2026-02-25T15:07:50.571571+0000","last_undegraded":"2026-02-25T15:07:50.571571+0000","last_fullsized":"2026-02-25T15:07:50.571571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:23:42.860020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8],"acting":[9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":621,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.565359+0000","last_change":"2026-02-25T15:02:14.166407+0000","last_active":"2026-02-25T15:07:50.565359+0000","last_peered":"2026-02-25T15:07:50.565359+0000","last_clean":"2026-02-25T15:07:50.565359+0000","last_became_active":"2026-02-25T15:02:14.164238+0000","last_became_peered":"2026-02-25T15:02:14.164238+0000","last_unstale":"2026-02-25T15:07:50.565359+0000","last_undegraded":"2026-02-25T15:07:50.565359+0000","last_fullsized":"2026-02-25T15:07:50.565359+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:06:11.961410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,15],"acting":[12,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.565158+0000","last_change":"2026-02-25T15:02:15.082986+0000","last_active":"2026-02-25T15:07:50.565158+0000","last_peered":"2026-02-25T15:07:50.565158+0000","last_clean":"2026-02-25T15:07:50.565158+0000","last_became_active":"2026-02-25T15:02:15.078177+0000","last_became_peered":"2026-02-25T15:02:15.078177+0000","last_unstale":"2026-02-25T15:07:50.565158+0000","last_undegraded":"2026-02-25T15:07:50.565158+0000","last_fullsized":"2026-02-25T15:07:50.565158+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:27:42.619819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10],"acting":[9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2026-02-25T15:07:50.563730+0000","last_change":"2026-02-25T15:02:14.239902+0000","last_active":"2026-02-25T15:07:50.563730+0000","last_peered":"2026-02-25T15:07:50.563730+0000","last_clean":"2026-02-25T15:07:50.563730+0000","last_became_active":"2026-02-25T15:02:14.237082+0000","last_became_peered":"2026-02-25T15:02:14.237082+0000","last_unstale":"2026-02-25T15:07:50.563730+0000","last_undegraded":"2026-02-25T15:07:50.563730+0000","last_fullsized":"2026-02-25T15:07:50.563730+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:57:36.384884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,14],"acting":[9,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":613,"reported_epoch":334,"state":"active+clean","last_fresh":"2026-02-25T15:07:48.437951+0000","last_change":"2026-02-25T15:02:14.132167+0000","last_active":"2026-02-25T15:07:48.437951+0000","last_peered":"2026-02-25T15:07:48.437951+0000","last_clean":"2026-02-25T15:07:48.437951+0000","last_became_active":"2026-02-25T15:02:14.116657+0000","last_became_peered":"2026-02-25T15:02:14.116657+0000","last_unstale":"2026-02-25T15:07:48.437951+0000","last_undegraded":"2026-02-25T15:07:48.437951+0000","last_fullsized":"2026-02-25T15:07:48.437951+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:23:36.296106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":16}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280591,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33644,"kb_used_data":6684,"kb_used_omap":5740,"kb_used_meta":21203,"kb_avail":94338196,"statfs":{"total":96636764160,"available":96602312704,"internally_reserved":0,"allocated":6844416,"data_stored":4222855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5878530,"internal_metadata":21712126},"hb_peers":[0,1,3,4,5,6,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280591,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6676,"kb_used_omap":203,"kb_used_meta":26740,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6836224,"data_stored":4222831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":208491,"internal_metadata":27382165},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280590,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34088,"kb_used_data":7128,"kb_used_omap":203,"kb_used_meta":26740,"kb_avail":94337752,"statfs":{"total":96636764160,"available":96601858048,"internally_reserved":0,"allocated":7299072,"data_stored":4682111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":208377,"internal_metadata":27382279},"hb_peers":[1,2,4,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313295,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51668,"kb_used_data":6660,"kb_used_omap":7601,"kb_used_meta":37390,"kb_avail":94320172,"statfs":{"total":96636764160,"available":96583856128,"internally_reserved":0,"allocated":6819840,"data_stored":4212890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7784319,"internal_metadata":38287489},"hb_peers":[0,2,3,4,5,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":14,"up_from":21,"seq":90194313294,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33604,"kb_used_data":6644,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338236,"statfs":{"total":96636764160,"available":96602353664,"internally_reserved":0,"allocated":6803456,"data_stored":4200771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210523,"internal_metadata":27380133},"hb_peers":[0,1,2,3,4,5,6,8,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313294,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33620,"kb_used_data":6660,"kb_used_omap":203,"kb_used_meta":26740,"kb_avail":94338220,"statfs":{"total":96636764160,"available":96602337280,"internally_reserved":0,"allocated":6819840,"data_stored":4212890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":207913,"internal_metadata":27382743},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313294,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33620,"kb_used_data":6660,"kb_used_omap":204,"kb_used_meta":26739,"kb_avail":94338220,"statfs":{"total":96636764160,"available":96602337280,"internally_reserved":0,"allocated":6819840,"data_stored":4212890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":209288,"internal_metadata":27381368},"hb_peers":[1,2,3,4,5,6,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313294,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33620,"kb_used_data":6660,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338220,"statfs":{"total":96636764160,"available":96602337280,"internally_reserved":0,"allocated":6819840,"data_stored":4212890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":209978,"internal_metadata":27380678},"hb_peers":[1,3,5,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313294,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33620,"kb_used_data":6660,"kb_used_omap":204,"kb_used_meta":26739,"kb_avail":94338220,"statfs":{"total":96636764160,"available":96602337280,"internally_reserved":0,"allocated":6819840,"data_stored":4212890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":209789,"internal_metadata":27380867},"hb_peers":[0,1,2,3,4,5,6,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313295,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6676,"kb_used_omap":203,"kb_used_meta":26740,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6836224,"data_stored":4222831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":208876,"internal_metadata":27381780},"hb_peers":[1,3,5,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313294,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6676,"kb_used_omap":206,"kb_used_meta":26737,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6836224,"data_stored":4222831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":211548,"internal_metadata":27379108},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313295,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34088,"kb_used_data":7128,"kb_used_omap":206,"kb_used_meta":26737,"kb_avail":94337752,"statfs":{"total":96636764160,"available":96601858048,"internally_reserved":0,"allocated":7299072,"data_stored":4682111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":211268,"internal_metadata":27379388},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313294,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6676,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6836224,"data_stored":4222831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210167,"internal_metadata":27380489},"hb_peers":[0,2,4,5,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313294,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33624,"kb_used_data":6664,"kb_used_omap":5825,"kb_used_meta":21118,"kb_avail":94338216,"statfs":{"total":96636764160,"available":96602333184,"internally_reserved":0,"allocated":6823936,"data_stored":4212895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5965730,"internal_metadata":21624926},"hb_peers":[0,1,2,4,5,6,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313295,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33624,"kb_used_data":6664,"kb_used_omap":203,"kb_used_meta":26740,"kb_avail":94338216,"statfs":{"total":96636764160,"available":96602333184,"internally_reserved":0,"allocated":6823936,"data_stored":4212909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":208021,"internal_metadata":27382635},"hb_peers":[0,1,2,3,4,5,7,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313295,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6676,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6836224,"data_stored":4222831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210527,"internal_metadata":27380129},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T15:07:53.048 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T15:07:53.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.113+0000 7f389c0ae640 10 monclient: get_monmap_and_config 2026-02-25T15:07:53.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.113+0000 7f389c0ae640 10 monclient: build_initial_monmap 2026-02-25T15:07:53.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.113+0000 7f389c0ae640 10 monclient: monmap: 2026-02-25T15:07:53.117 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:53.117 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:53.117 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:53.115425+0000 2026-02-25T15:07:53.117 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:53.115425+0000 2026-02-25T15:07:53.117 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:53.117 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:53.117 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:53.118 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:53.118 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:53.118 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:53.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.114+0000 7f389c0ae640 1 Processor -- start 2026-02-25T15:07:53.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.114+0000 7f389c0ae640 1 -- start start 2026-02-25T15:07:53.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.114+0000 7f389c0ae640 10 monclient: init 2026-02-25T15:07:53.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.114+0000 7f389c0ae640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:53.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.114+0000 7f389c0ae640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:07:53.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.114+0000 7f389c0ae640 10 monclient(hunting): picked mon.noname-b con 0x7f3894058da0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:53.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.114+0000 7f389c0ae640 10 monclient(hunting): picked mon.noname-c con 0x7f38940b0410 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:53.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.114+0000 7f389c0ae640 10 monclient(hunting): picked mon.noname-a con 0x7f38941626a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:53.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.114+0000 7f389c0ae640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38941655c0 con 0x7f38941626a0 2026-02-25T15:07:53.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.114+0000 7f389c0ae640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38940b12b0 con 0x7f3894058da0 2026-02-25T15:07:53.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.114+0000 7f389c0ae640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38940b1590 con 0x7f38940b0410 2026-02-25T15:07:53.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.114+0000 7f389c0ae640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:53.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.114+0000 7f389c0ae640 10 monclient(hunting): authenticate will time out at 3362.715776s 2026-02-25T15:07:53.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.115+0000 7f389a624640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f38941626a0 0x7f38941c6d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:32956/0 (socket says 10.20.193.90:32956) 2026-02-25T15:07:53.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.115+0000 7f389a624640 1 -- 10.20.193.90:0/101066311 learned_addr learned my addr 10.20.193.90:0/101066311 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:53.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 1 -- 10.20.193.90:0/101066311 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3223545012 0 0) 0x7f38941655c0 con 0x7f38941626a0 2026-02-25T15:07:53.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 10 monclient(hunting): _negotiate 2026-02-25T15:07:53.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:53.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:53.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 10 monclient(hunting): my global_id is 30777 2026-02-25T15:07:53.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 1 -- 10.20.193.90:0/101066311 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f387c003610 con 0x7f38941626a0 2026-02-25T15:07:53.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 1 -- 10.20.193.90:0/101066311 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2962734104 0 0) 0x7f38940b1590 con 0x7f38940b0410 2026-02-25T15:07:53.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 10 monclient(hunting): _negotiate 2026-02-25T15:07:53.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:53.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:53.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 10 monclient(hunting): my global_id is 28547 2026-02-25T15:07:53.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 1 -- 10.20.193.90:0/101066311 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38941655c0 con 0x7f38940b0410 2026-02-25T15:07:53.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 1 -- 10.20.193.90:0/101066311 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2832339139 0 0) 0x7f38940b12b0 con 0x7f3894058da0 2026-02-25T15:07:53.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 10 monclient(hunting): _negotiate 2026-02-25T15:07:53.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:53.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:53.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 10 monclient(hunting): my global_id is 27409 2026-02-25T15:07:53.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.116+0000 7f3898e21640 1 -- 10.20.193.90:0/101066311 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38940b1590 con 0x7f3894058da0 2026-02-25T15:07:53.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f3898e21640 1 -- 10.20.193.90:0/101066311 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1059619112 0 0) 0x7f387c003610 con 0x7f38941626a0 2026-02-25T15:07:53.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f3898e21640 1 -- 10.20.193.90:0/101066311 >> v1:10.20.193.94:6789/0 conn(0x7f38940b0410 legacy=0x7f3894164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:53.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f3898e21640 1 -- 10.20.193.90:0/101066311 >> v1:10.20.193.93:6789/0 conn(0x7f3894058da0 legacy=0x7f38941647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:53.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f3898e21640 10 monclient: _finish_hunting 0 2026-02-25T15:07:53.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f3898e21640 1 monclient: found mon.noname-a 2026-02-25T15:07:53.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f3898e21640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:53.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f3898e21640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:07:53.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f3898e21640 1 -- 10.20.193.90:0/101066311 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38940b1870 con 0x7f38941626a0 2026-02-25T15:07:53.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f3898e21640 10 monclient: _finish_auth 0 2026-02-25T15:07:53.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f3898e21640 10 monclient: _check_auth_tickets 2026-02-25T15:07:53.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f3898e21640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:53.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f3898e21640 1 -- 10.20.193.90:0/101066311 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3890003170 con 0x7f38941626a0 2026-02-25T15:07:53.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f3898e21640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:53.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f3898e21640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:53.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f3898e21640 10 monclient: dump: 2026-02-25T15:07:53.125 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:53.125 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:53.125 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:53.125 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:53.125 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:53.125 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:53.125 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:53.125 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:53.125 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:53.125 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:53.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f389c0ae640 5 monclient: authenticate success, global_id 30777 2026-02-25T15:07:53.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.118+0000 7f389c0ae640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:53.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.119+0000 7f3898e21640 1 -- 10.20.193.90:0/101066311 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3890003340 con 0x7f38941626a0 2026-02-25T15:07:53.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.119+0000 7f3898e21640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:53.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.119+0000 7f3898e21640 1 -- 10.20.193.90:0/101066311 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3890003780 con 0x7f38941626a0 2026-02-25T15:07:53.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.119+0000 7f3898e21640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:53.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.119+0000 7f3898e21640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:53.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.119+0000 7f3898e21640 10 monclient: dump: 2026-02-25T15:07:53.132 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:53.132 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:53.132 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:53.132 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:53.132 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:53.132 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:53.132 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:53.133 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:53.133 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:53.133 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:53.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.119+0000 7f389c0ae640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:53.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.119+0000 7f389c0ae640 4 set_mon_vals no callback set 2026-02-25T15:07:53.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.119+0000 7f389c0ae640 10 monclient: shutdown 2026-02-25T15:07:53.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.119+0000 7f389c0ae640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:53.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.119+0000 7f389c0ae640 1 -- 10.20.193.90:0/101066311 >> v1:10.20.193.90:6789/0 conn(0x7f38941626a0 legacy=0x7f38941c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:53.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.119+0000 7f389c0ae640 1 -- 10.20.193.90:0/101066311 shutdown_connections 2026-02-25T15:07:53.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.119+0000 7f389c0ae640 1 -- 10.20.193.90:0/101066311 >> 10.20.193.90:0/101066311 conn(0x7f38940b65f0 msgr2=0x7f38940b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:53.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.119+0000 7f389c0ae640 1 -- 10.20.193.90:0/101066311 shutdown_connections 2026-02-25T15:07:53.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.119+0000 7f389c0ae640 1 -- 10.20.193.90:0/101066311 wait complete. 2026-02-25T15:07:53.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 10 monclient: build_initial_monmap 2026-02-25T15:07:53.133 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 10 monclient: monmap: 2026-02-25T15:07:53.134 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:53.134 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:53.134 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:53.134 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:53.134 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:53.134 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:53.134 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:53.134 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:53.134 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:53.134 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:53.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 1 librados: starting msgr at 2026-02-25T15:07:53.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 1 librados: starting objecter 2026-02-25T15:07:53.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:53.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 1 Processor -- start 2026-02-25T15:07:53.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 1 -- start start 2026-02-25T15:07:53.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 1 librados: setting wanted keys 2026-02-25T15:07:53.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 1 librados: calling monclient init 2026-02-25T15:07:53.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 10 monclient: init 2026-02-25T15:07:53.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:53.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:07:53.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 10 monclient(hunting): picked mon.noname-a con 0x7f3894058da0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:53.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 10 monclient(hunting): picked mon.noname-b con 0x7f38940b0410 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:53.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 10 monclient(hunting): picked mon.noname-c con 0x7f38941626a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:53.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3894132a80 con 0x7f3894058da0 2026-02-25T15:07:53.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f389405be60 con 0x7f38940b0410 2026-02-25T15:07:53.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f389405c140 con 0x7f38941626a0 2026-02-25T15:07:53.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:53.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.120+0000 7f389c0ae640 10 monclient(hunting): authenticate will time out at 3362.721801s 2026-02-25T15:07:53.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.121+0000 7f3899e23640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f3894058da0 0x7f3894138ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:32962/0 (socket says 10.20.193.90:32962) 2026-02-25T15:07:53.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.121+0000 7f3899e23640 1 -- 10.20.193.90:0/1568979717 learned_addr learned my addr 10.20.193.90:0/1568979717 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:53.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.121+0000 7f38827fc640 10 client.?.objecter ms_handle_connect 0x7f3894058da0 2026-02-25T15:07:53.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.121+0000 7f38827fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:53.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.121+0000 7f38827fc640 10 client.?.objecter ms_handle_connect 0x7f38940b0410 2026-02-25T15:07:53.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.121+0000 7f38827fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:53.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.121+0000 7f38827fc640 10 client.?.objecter ms_handle_connect 0x7f38941626a0 2026-02-25T15:07:53.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.121+0000 7f38827fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:53.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1298613998 0 0) 0x7f3894132a80 con 0x7f3894058da0 2026-02-25T15:07:53.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:53.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:53.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:53.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 10 monclient(hunting): my global_id is 30780 2026-02-25T15:07:53.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3864003610 con 0x7f3894058da0 2026-02-25T15:07:53.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3023910156 0 0) 0x7f389405be60 con 0x7f38940b0410 2026-02-25T15:07:53.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:53.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:53.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:53.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 10 monclient(hunting): my global_id is 27412 2026-02-25T15:07:53.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3894132a80 con 0x7f38940b0410 2026-02-25T15:07:53.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1035940001 0 0) 0x7f389405c140 con 0x7f38941626a0 2026-02-25T15:07:53.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:53.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:53.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:53.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 10 monclient(hunting): my global_id is 28550 2026-02-25T15:07:53.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.122+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f389405be60 con 0x7f38941626a0 2026-02-25T15:07:53.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.124+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2217317939 0 0) 0x7f3864003610 con 0x7f3894058da0 2026-02-25T15:07:53.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.124+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f389405c140 con 0x7f3894058da0 2026-02-25T15:07:53.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.124+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3884003170 con 0x7f3894058da0 2026-02-25T15:07:53.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.124+0000 7f38827fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:53.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.124+0000 7f38827fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:53.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.124+0000 7f38827fc640 10 monclient(hunting): dump: 2026-02-25T15:07:53.138 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:53.138 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:53.138 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:53.138 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:53.138 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:53.138 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:53.138 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:53.138 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:53.139 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:53.139 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:53.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.124+0000 7f38827fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:53.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.124+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3029495136 0 0) 0x7f3894132a80 con 0x7f38940b0410 2026-02-25T15:07:53.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.124+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3864003610 con 0x7f38940b0410 2026-02-25T15:07:53.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.124+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f38880030b0 con 0x7f38940b0410 2026-02-25T15:07:53.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.124+0000 7f38827fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:53.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.124+0000 7f38827fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:53.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.124+0000 7f38827fc640 10 monclient(hunting): dump: 2026-02-25T15:07:53.139 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:53.139 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:53.139 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:53.139 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:53.139 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:53.140 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:53.140 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:53.140 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:53.140 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:53.140 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:53.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.124+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2544448323 0 0) 0x7f389405be60 con 0x7f38941626a0 2026-02-25T15:07:53.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.125+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3894132a80 con 0x7f38941626a0 2026-02-25T15:07:53.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.125+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f38900034f0 con 0x7f38941626a0 2026-02-25T15:07:53.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.125+0000 7f38827fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:53.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.125+0000 7f38827fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:53.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.125+0000 7f38827fc640 10 monclient(hunting): dump: 2026-02-25T15:07:53.140 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:53.140 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:53.141 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:53.141 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:53.141 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:53.141 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:53.141 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:53.141 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:53.141 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:53.141 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:53.141 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3776185672 0 0) 0x7f389405c140 con 0x7f3894058da0 2026-02-25T15:07:53.141 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 >> v1:10.20.193.94:6789/0 conn(0x7f38941626a0 legacy=0x7f3894260150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:53.141 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 >> v1:10.20.193.93:6789/0 conn(0x7f38940b0410 legacy=0x7f3894132370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:53.141 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f38827fc640 10 monclient: _finish_hunting 0 2026-02-25T15:07:53.141 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f38827fc640 1 monclient: found mon.a 2026-02-25T15:07:53.141 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f38827fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:53.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f38827fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:53.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389405c420 con 0x7f3894058da0 2026-02-25T15:07:53.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f38827fc640 10 monclient: _finish_auth 0 2026-02-25T15:07:53.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f38827fc640 10 monclient: _check_auth_tickets 2026-02-25T15:07:53.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f38827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:53.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f389c0ae640 5 monclient: authenticate success, global_id 30780 2026-02-25T15:07:53.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f389c0ae640 10 monclient: _renew_subs 2026-02-25T15:07:53.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f389c0ae640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:53.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f389c0ae640 1 -- 10.20.193.90:0/1568979717 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f389405c700 con 0x7f3894058da0 2026-02-25T15:07:53.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f389c0ae640 10 client.30780.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:53.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f389c0ae640 10 monclient: _renew_subs 2026-02-25T15:07:53.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f389c0ae640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:53.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f389c0ae640 1 -- 10.20.193.90:0/1568979717 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f389405bae0 con 0x7f3894058da0 2026-02-25T15:07:53.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f389c0ae640 1 librados: init done 2026-02-25T15:07:53.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f389c0ae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:53.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f389c0ae640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:53.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f389c0ae640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:53.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.126+0000 7f389c0ae640 1 -- 10.20.193.90:0/1568979717 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38941640c0 con 0x7f3894058da0 2026-02-25T15:07:53.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.216+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3884003310 con 0x7f3894058da0 2026-02-25T15:07:53.219 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.216+0000 7f38827fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:53.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.216+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3884003b50 con 0x7f3894058da0 2026-02-25T15:07:53.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.216+0000 7f38827fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:53.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.216+0000 7f38827fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:53.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.216+0000 7f3883fff640 4 set_mon_vals no callback set 2026-02-25T15:07:53.220 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.216+0000 7f38827fc640 10 monclient: dump: 2026-02-25T15:07:53.220 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:53.220 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:53.220 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:53.220 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:53.220 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:53.220 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:53.220 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:53.221 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:53.221 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:53.221 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:53.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.217+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f3884003d30 con 0x7f3894058da0 2026-02-25T15:07:53.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.218+0000 7f3899622640 10 monclient: get_auth_request con 0x7f386407a090 auth_method 0 2026-02-25T15:07:53.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.218+0000 7f38827fc640 10 client.30780.objecter ms_handle_connect 0x7f386407a090 2026-02-25T15:07:53.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.219+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(337..337 src has 1..337) ==== 8959+0+0 (unknown 3556021173 0 0) 0x7f3884095cf0 con 0x7f3894058da0 2026-02-25T15:07:53.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.219+0000 7f38827fc640 10 client.30780.objecter ms_dispatch2 0x7f3894000b90 osd_map(337..337 src has 1..337) 2026-02-25T15:07:53.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.219+0000 7f38827fc640 3 client.30780.objecter handle_osd_map got epochs [337,337] > 0 2026-02-25T15:07:53.222 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.219+0000 7f38827fc640 3 client.30780.objecter handle_osd_map decoding full epoch 337 2026-02-25T15:07:53.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.219+0000 7f38827fc640 20 client.30780.objecter dump_active .. 0 homeless 2026-02-25T15:07:53.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.219+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f38941640c0 con 0x7f3894058da0 2026-02-25T15:07:53.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.219+0000 7f38827fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:53.223 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.219+0000 7f38827fc640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:53.327 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.324+0000 7f389c0ae640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:07:53.327 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.324+0000 7f389c0ae640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:07:53.327 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.324+0000 7f389c0ae640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:53.327 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.324+0000 7f389c0ae640 1 -- 10.20.193.90:0/1568979717 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38940b0c40 con 0x7f3894058da0 2026-02-25T15:07:53.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.339+0000 7f38827fc640 1 -- 10.20.193.90:0/1568979717 <== mon.0 v1:10.20.193.90:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v337) ==== 74+0+23427 (unknown 410704729 0 731378157) 0x7f38940b0c40 con 0x7f3894058da0 2026-02-25T15:07:53.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.339+0000 7f38827fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-25T15:07:53.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.339+0000 7f38827fc640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:07:53.342 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:07:53.343 INFO:teuthology.orchestra.run.trial090.stdout:{"epoch":337,"fsid":"63c679b5-d219-4797-8adb-7340f3fd6819","created":"2026-02-25T15:00:56.256818+0000","modified":"2026-02-25T15:07:51.557116+0000","last_up_change":"2026-02-25T15:01:29.405841+0000","last_in_change":"2026-02-25T15:01:02.711189+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":94,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":268,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T15:01:29.228894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-25T15:01:37.037528+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":5.9899997711181641,"score_stable":5.9899997711181641,"optimal_score":0.68999999761581421,"raw_score_acting":4.130000114440918,"raw_score_stable":4.130000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"ceph_test_rados_api_asio","create_time":"2026-02-25T15:02:11.172368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ad97006e-f5c3-4e36-97a0-7fa2b250344d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":290,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6808","nonce":3221464204}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6809","nonce":3221464204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6811","nonce":3221464204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6810","nonce":3221464204}]},"public_addr":"10.20.193.90:6808/3221464204","cluster_addr":"10.20.193.90:6809/3221464204","heartbeat_back_addr":"10.20.193.90:6811/3221464204","heartbeat_front_addr":"10.20.193.90:6810/3221464204","state":["exists","up"]},{"osd":1,"uuid":"25b02f30-96b9-43a3-9691-f06a1292e36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":335,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6804","nonce":3161035071}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6805","nonce":3161035071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6807","nonce":3161035071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6806","nonce":3161035071}]},"public_addr":"10.20.193.93:6804/3161035071","cluster_addr":"10.20.193.93:6805/3161035071","heartbeat_back_addr":"10.20.193.93:6807/3161035071","heartbeat_front_addr":"10.20.193.93:6806/3161035071","state":["exists","up"]},{"osd":2,"uuid":"aeed908b-d483-498b-83ba-632e51dd05e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":335,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6800","nonce":3705139403}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6801","nonce":3705139403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6803","nonce":3705139403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6802","nonce":3705139403}]},"public_addr":"10.20.193.94:6800/3705139403","cluster_addr":"10.20.193.94:6801/3705139403","heartbeat_back_addr":"10.20.193.94:6803/3705139403","heartbeat_front_addr":"10.20.193.94:6802/3705139403","state":["exists","up"]},{"osd":3,"uuid":"c7a30346-685f-4bd2-acaa-00b98ad3284c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":335,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6813","nonce":2702305154}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6814","nonce":2702305154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6816","nonce":2702305154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6815","nonce":2702305154}]},"public_addr":"10.20.193.189:6813/2702305154","cluster_addr":"10.20.193.189:6814/2702305154","heartbeat_back_addr":"10.20.193.189:6816/2702305154","heartbeat_front_addr":"10.20.193.189:6815/2702305154","state":["exists","up"]},{"osd":4,"uuid":"d6c97301-55cc-4579-bc92-cb240c5ebe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":335,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6800","nonce":1446526134}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6802","nonce":1446526134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6805","nonce":1446526134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6803","nonce":1446526134}]},"public_addr":"10.20.193.90:6800/1446526134","cluster_addr":"10.20.193.90:6802/1446526134","heartbeat_back_addr":"10.20.193.90:6805/1446526134","heartbeat_front_addr":"10.20.193.90:6803/1446526134","state":["exists","up"]},{"osd":5,"uuid":"e3117058-2f0c-466b-a302-061fc4c120fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":335,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6800","nonce":887686167}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6801","nonce":887686167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6803","nonce":887686167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6802","nonce":887686167}]},"public_addr":"10.20.193.93:6800/887686167","cluster_addr":"10.20.193.93:6801/887686167","heartbeat_back_addr":"10.20.193.93:6803/887686167","heartbeat_front_addr":"10.20.193.93:6802/887686167","state":["exists","up"]},{"osd":6,"uuid":"f4b4a3ce-0a0b-4386-a433-dcbf65558ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":335,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6812","nonce":1114158993}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6813","nonce":1114158993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6815","nonce":1114158993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6814","nonce":1114158993}]},"public_addr":"10.20.193.94:6812/1114158993","cluster_addr":"10.20.193.94:6813/1114158993","heartbeat_back_addr":"10.20.193.94:6815/1114158993","heartbeat_front_addr":"10.20.193.94:6814/1114158993","state":["exists","up"]},{"osd":7,"uuid":"3337e549-6b6e-444e-997b-18a19beb5640","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":335,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6809","nonce":1708410075}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6810","nonce":1708410075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6812","nonce":1708410075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6811","nonce":1708410075}]},"public_addr":"10.20.193.189:6809/1708410075","cluster_addr":"10.20.193.189:6810/1708410075","heartbeat_back_addr":"10.20.193.189:6812/1708410075","heartbeat_front_addr":"10.20.193.189:6811/1708410075","state":["exists","up"]},{"osd":8,"uuid":"b292f7c4-9a41-4e48-876c-29281c5ec73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":335,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6801","nonce":2320036239}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6804","nonce":2320036239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6807","nonce":2320036239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6806","nonce":2320036239}]},"public_addr":"10.20.193.90:6801/2320036239","cluster_addr":"10.20.193.90:6804/2320036239","heartbeat_back_addr":"10.20.193.90:6807/2320036239","heartbeat_front_addr":"10.20.193.90:6806/2320036239","state":["exists","up"]},{"osd":9,"uuid":"954a7cbb-2f06-4cb9-ae68-380ac58a9f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":335,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6808","nonce":3039017972}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6809","nonce":3039017972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6811","nonce":3039017972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6810","nonce":3039017972}]},"public_addr":"10.20.193.93:6808/3039017972","cluster_addr":"10.20.193.93:6809/3039017972","heartbeat_back_addr":"10.20.193.93:6811/3039017972","heartbeat_front_addr":"10.20.193.93:6810/3039017972","state":["exists","up"]},{"osd":10,"uuid":"95ed1c74-8ca5-4317-9ef7-fc657b3cda70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":335,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6804","nonce":4185080112}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6805","nonce":4185080112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6807","nonce":4185080112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6806","nonce":4185080112}]},"public_addr":"10.20.193.94:6804/4185080112","cluster_addr":"10.20.193.94:6805/4185080112","heartbeat_back_addr":"10.20.193.94:6807/4185080112","heartbeat_front_addr":"10.20.193.94:6806/4185080112","state":["exists","up"]},{"osd":11,"uuid":"8f617b59-e8d1-42fb-8787-64e8c2117954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":335,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6801","nonce":4031575111}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6802","nonce":4031575111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6804","nonce":4031575111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6803","nonce":4031575111}]},"public_addr":"10.20.193.189:6801/4031575111","cluster_addr":"10.20.193.189:6802/4031575111","heartbeat_back_addr":"10.20.193.189:6804/4031575111","heartbeat_front_addr":"10.20.193.189:6803/4031575111","state":["exists","up"]},{"osd":12,"uuid":"4bad4371-2ca7-4023-8406-0275011eda9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":335,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6812","nonce":1139796254}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6813","nonce":1139796254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6815","nonce":1139796254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.90:6814","nonce":1139796254}]},"public_addr":"10.20.193.90:6812/1139796254","cluster_addr":"10.20.193.90:6813/1139796254","heartbeat_back_addr":"10.20.193.90:6815/1139796254","heartbeat_front_addr":"10.20.193.90:6814/1139796254","state":["exists","up"]},{"osd":13,"uuid":"5d1ca952-93a5-43f3-9780-8bc85f9ed3d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":335,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6812","nonce":1850919728}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6813","nonce":1850919728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6815","nonce":1850919728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.93:6814","nonce":1850919728}]},"public_addr":"10.20.193.93:6812/1850919728","cluster_addr":"10.20.193.93:6813/1850919728","heartbeat_back_addr":"10.20.193.93:6815/1850919728","heartbeat_front_addr":"10.20.193.93:6814/1850919728","state":["exists","up"]},{"osd":14,"uuid":"50853d20-2756-4720-b7aa-3020ded29189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":335,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6808","nonce":887271598}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6809","nonce":887271598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6811","nonce":887271598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.94:6810","nonce":887271598}]},"public_addr":"10.20.193.94:6808/887271598","cluster_addr":"10.20.193.94:6809/887271598","heartbeat_back_addr":"10.20.193.94:6811/887271598","heartbeat_front_addr":"10.20.193.94:6810/887271598","state":["exists","up"]},{"osd":15,"uuid":"4445eda7-fb77-4f2c-8fda-77667df22b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":335,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6805","nonce":4039858964}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6806","nonce":4039858964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6808","nonce":4039858964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.189:6807","nonce":4039858964}]},"public_addr":"10.20.193.189:6805/4039858964","cluster_addr":"10.20.193.189:6806/4039858964","heartbeat_back_addr":"10.20.193.189:6808/4039858964","heartbeat_front_addr":"10.20.193.189:6807/4039858964","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.787596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.833159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.652364+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:27.190420+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.727824+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.707073+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.887715+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:27.132243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.999569+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.929271+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.761416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.870660+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.873627+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:27.124178+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.868032+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T15:01:26.940190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":10,"snaps":[{"begin":1,"length":4}]},{"pool":11,"snaps":[{"begin":2,"length":5}]},{"pool":32,"snaps":[{"begin":1,"length":1}]},{"pool":60,"snaps":[{"begin":1,"length":1}]},{"pool":92,"snaps":[{"begin":7,"length":2}]},{"pool":100,"snaps":[{"begin":1,"length":1}]},{"pool":120,"snaps":[{"begin":1,"length":2}]},{"pool":137,"snaps":[{"begin":3,"length":1}]},{"pool":147,"snaps":[{"begin":1,"length":4}]},{"pool":148,"snaps":[{"begin":2,"length":2}]},{"pool":154,"snaps":[{"begin":2,"length":3}]},{"pool":166,"snaps":[{"begin":2,"length":2}]},{"pool":170,"snaps":[{"begin":2,"length":4}]},{"pool":177,"snaps":[{"begin":2,"length":1}]},{"pool":183,"snaps":[{"begin":1,"length":4}]},{"pool":194,"snaps":[{"begin":4,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T15:07:53.344 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.341+0000 7f389c0ae640 10 librados: watch_flush enter 2026-02-25T15:07:53.345 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.341+0000 7f389c0ae640 10 librados: watch_flush exit 2026-02-25T15:07:53.345 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.341+0000 7f389c0ae640 20 client.30780.objecter shutdown clearing up homeless session... 2026-02-25T15:07:53.345 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.341+0000 7f389c0ae640 10 client.30780.objecter successfully canceled tick 2026-02-25T15:07:53.345 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.341+0000 7f389c0ae640 1 -- 10.20.193.90:0/1568979717 >> v1:10.20.193.189:6800/3647375982 conn(0x7f386407a090 legacy=0x7f386409a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:53.345 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.341+0000 7f389c0ae640 10 monclient: shutdown 2026-02-25T15:07:53.345 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.341+0000 7f389c0ae640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:53.345 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.341+0000 7f389c0ae640 1 -- 10.20.193.90:0/1568979717 >> v1:10.20.193.90:6789/0 conn(0x7f3894058da0 legacy=0x7f3894138ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:53.345 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.341+0000 7f389c0ae640 1 -- 10.20.193.90:0/1568979717 shutdown_connections 2026-02-25T15:07:53.345 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.341+0000 7f389c0ae640 1 -- 10.20.193.90:0/1568979717 >> 10.20.193.90:0/1568979717 conn(0x7f38940b65f0 msgr2=0x7f3894055aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:53.346 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.341+0000 7f389c0ae640 1 -- 10.20.193.90:0/1568979717 shutdown_connections 2026-02-25T15:07:53.346 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.341+0000 7f389c0ae640 1 -- 10.20.193.90:0/1568979717 wait complete. 2026-02-25T15:07:53.346 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:53.341+0000 7f389c0ae640 1 librados: shutdown 2026-02-25T15:07:54.355 INFO:tasks.ceph:Scrubbing osd.0 2026-02-25T15:07:54.356 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:07:54.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.426+0000 7f066f283640 10 monclient: get_monmap_and_config 2026-02-25T15:07:54.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.426+0000 7f066f283640 10 monclient: build_initial_monmap 2026-02-25T15:07:54.429 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.426+0000 7f066f283640 10 monclient: monmap: 2026-02-25T15:07:54.429 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:54.429 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.430 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:54.427857+0000 2026-02-25T15:07:54.430 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:54.427857+0000 2026-02-25T15:07:54.430 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:54.430 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.430 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:54.430 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:54.430 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:54.430 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.426+0000 7f066f283640 1 Processor -- start 2026-02-25T15:07:54.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.426+0000 7f066f283640 1 -- start start 2026-02-25T15:07:54.430 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.426+0000 7f066f283640 10 monclient: init 2026-02-25T15:07:54.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.427+0000 7f066f283640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:54.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.427+0000 7f066f283640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:07:54.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.427+0000 7f066f283640 10 monclient(hunting): picked mon.noname-a con 0x7f066815e2a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:54.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.427+0000 7f066f283640 10 monclient(hunting): picked mon.noname-b con 0x7f0668059e10 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:54.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.427+0000 7f066f283640 10 monclient(hunting): picked mon.noname-c con 0x7f06681a5660 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:54.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.427+0000 7f066f283640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0668162df0 con 0x7f066815e2a0 2026-02-25T15:07:54.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.427+0000 7f066f283640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06681630d0 con 0x7f0668059e10 2026-02-25T15:07:54.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.427+0000 7f066f283640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06681633b0 con 0x7f06681a5660 2026-02-25T15:07:54.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.427+0000 7f066f283640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:54.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.427+0000 7f066f283640 10 monclient(hunting): authenticate will time out at 3364.028426s 2026-02-25T15:07:54.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.427+0000 7f066cff8640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f066815e2a0 0x7f066805da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39590/0 (socket says 10.20.193.90:39590) 2026-02-25T15:07:54.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.427+0000 7f066cff8640 1 -- 10.20.193.90:0/1729169237 learned_addr learned my addr 10.20.193.90:0/1729169237 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:54.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 1 -- 10.20.193.90:0/1729169237 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 600189733 0 0) 0x7f0668162df0 con 0x7f066815e2a0 2026-02-25T15:07:54.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 10 monclient(hunting): my global_id is 30783 2026-02-25T15:07:54.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 1 -- 10.20.193.90:0/1729169237 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0654003610 con 0x7f066815e2a0 2026-02-25T15:07:54.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 1 -- 10.20.193.90:0/1729169237 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1222937447 0 0) 0x7f06681630d0 con 0x7f0668059e10 2026-02-25T15:07:54.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 10 monclient(hunting): my global_id is 27415 2026-02-25T15:07:54.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 1 -- 10.20.193.90:0/1729169237 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0668162df0 con 0x7f0668059e10 2026-02-25T15:07:54.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 1 -- 10.20.193.90:0/1729169237 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 364758113 0 0) 0x7f06681633b0 con 0x7f06681a5660 2026-02-25T15:07:54.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 10 monclient(hunting): my global_id is 28553 2026-02-25T15:07:54.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.429+0000 7f065f7fe640 1 -- 10.20.193.90:0/1729169237 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06681630d0 con 0x7f06681a5660 2026-02-25T15:07:54.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f065f7fe640 1 -- 10.20.193.90:0/1729169237 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3066942350 0 0) 0x7f0654003610 con 0x7f066815e2a0 2026-02-25T15:07:54.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f065f7fe640 1 -- 10.20.193.90:0/1729169237 >> v1:10.20.193.94:6789/0 conn(0x7f06681a5660 legacy=0x7f06681c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f065f7fe640 1 -- 10.20.193.90:0/1729169237 >> v1:10.20.193.93:6789/0 conn(0x7f0668059e10 legacy=0x7f066815ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f065f7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:07:54.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f065f7fe640 1 monclient: found mon.noname-a 2026-02-25T15:07:54.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f065f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:54.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f065f7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:07:54.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f065f7fe640 1 -- 10.20.193.90:0/1729169237 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0668163690 con 0x7f066815e2a0 2026-02-25T15:07:54.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f065f7fe640 10 monclient: _finish_auth 0 2026-02-25T15:07:54.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f065f7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:07:54.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f065f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:54.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f065f7fe640 1 -- 10.20.193.90:0/1729169237 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0658002df0 con 0x7f066815e2a0 2026-02-25T15:07:54.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f065f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:54.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f065f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:54.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f065f7fe640 10 monclient: dump: 2026-02-25T15:07:54.437 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.437 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.437 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.437 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.437 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.438 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.438 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.438 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.438 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.438 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f066f283640 5 monclient: authenticate success, global_id 30783 2026-02-25T15:07:54.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.430+0000 7f066f283640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:54.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.431+0000 7f065f7fe640 1 -- 10.20.193.90:0/1729169237 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0658002fc0 con 0x7f066815e2a0 2026-02-25T15:07:54.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.431+0000 7f065f7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:54.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.431+0000 7f065f7fe640 1 -- 10.20.193.90:0/1729169237 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0658003400 con 0x7f066815e2a0 2026-02-25T15:07:54.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.431+0000 7f065f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:54.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.431+0000 7f065f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:54.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.431+0000 7f065f7fe640 10 monclient: dump: 2026-02-25T15:07:54.444 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.444 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.444 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.444 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.444 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.445 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.445 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.445 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.445 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.445 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.431+0000 7f066f283640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:54.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.431+0000 7f066f283640 4 set_mon_vals no callback set 2026-02-25T15:07:54.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 10 monclient: shutdown 2026-02-25T15:07:54.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:54.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 1 -- 10.20.193.90:0/1729169237 >> v1:10.20.193.90:6789/0 conn(0x7f066815e2a0 legacy=0x7f066805da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 1 -- 10.20.193.90:0/1729169237 shutdown_connections 2026-02-25T15:07:54.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 1 -- 10.20.193.90:0/1729169237 >> 10.20.193.90:0/1729169237 conn(0x7f06680b2f60 msgr2=0x7f06680b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:54.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 1 -- 10.20.193.90:0/1729169237 shutdown_connections 2026-02-25T15:07:54.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 1 -- 10.20.193.90:0/1729169237 wait complete. 2026-02-25T15:07:54.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 10 monclient: build_initial_monmap 2026-02-25T15:07:54.446 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 10 monclient: monmap: 2026-02-25T15:07:54.446 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:54.446 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:54.446 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:54.446 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:54.446 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:54.446 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:54.446 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:54.447 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:54.447 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:54.447 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 1 librados: starting msgr at 2026-02-25T15:07:54.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 1 librados: starting objecter 2026-02-25T15:07:54.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:54.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 1 Processor -- start 2026-02-25T15:07:54.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 1 -- start start 2026-02-25T15:07:54.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 1 librados: setting wanted keys 2026-02-25T15:07:54.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 1 librados: calling monclient init 2026-02-25T15:07:54.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.432+0000 7f066f283640 10 monclient: init 2026-02-25T15:07:54.447 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f066f283640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:54.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f066f283640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:07:54.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f066f283640 10 monclient(hunting): picked mon.noname-b con 0x7f0668059e10 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:54.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f066f283640 10 monclient(hunting): picked mon.noname-c con 0x7f066815e2a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:54.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f066f283640 10 monclient(hunting): picked mon.noname-a con 0x7f06681a5660 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:54.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f066f283640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f066805c560 con 0x7f06681a5660 2026-02-25T15:07:54.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f066f283640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f066815f9f0 con 0x7f0668059e10 2026-02-25T15:07:54.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f066f283640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f066815fcd0 con 0x7f066815e2a0 2026-02-25T15:07:54.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f066f283640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:54.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f066f283640 10 monclient(hunting): authenticate will time out at 3364.034387s 2026-02-25T15:07:54.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f066d7f9640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f06681a5660 0x7f0668056a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39594/0 (socket says 10.20.193.90:39594) 2026-02-25T15:07:54.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f066d7f9640 1 -- 10.20.193.90:0/2499429412 learned_addr learned my addr 10.20.193.90:0/2499429412 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:54.448 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f065d7fa640 10 client.?.objecter ms_handle_connect 0x7f06681a5660 2026-02-25T15:07:54.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f065d7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:54.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f065d7fa640 10 client.?.objecter ms_handle_connect 0x7f0668059e10 2026-02-25T15:07:54.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.433+0000 7f065d7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:54.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.434+0000 7f065d7fa640 10 client.?.objecter ms_handle_connect 0x7f066815e2a0 2026-02-25T15:07:54.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.434+0000 7f065d7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:54.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.434+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1024208677 0 0) 0x7f066805c560 con 0x7f06681a5660 2026-02-25T15:07:54.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.434+0000 7f065d7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.434+0000 7f065d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.434+0000 7f065d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.434+0000 7f065d7fa640 10 monclient(hunting): my global_id is 30786 2026-02-25T15:07:54.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.434+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0638003610 con 0x7f06681a5660 2026-02-25T15:07:54.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.435+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3349497033 0 0) 0x7f066815fcd0 con 0x7f066815e2a0 2026-02-25T15:07:54.449 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.435+0000 7f065d7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.435+0000 7f065d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.435+0000 7f065d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.435+0000 7f065d7fa640 10 monclient(hunting): my global_id is 28556 2026-02-25T15:07:54.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.435+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f066805c560 con 0x7f066815e2a0 2026-02-25T15:07:54.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.435+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1271603429 0 0) 0x7f066815f9f0 con 0x7f0668059e10 2026-02-25T15:07:54.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.435+0000 7f065d7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.435+0000 7f065d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.435+0000 7f065d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.435+0000 7f065d7fa640 10 monclient(hunting): my global_id is 27418 2026-02-25T15:07:54.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.435+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f066815fcd0 con 0x7f0668059e10 2026-02-25T15:07:54.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.436+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 223744116 0 0) 0x7f0638003610 con 0x7f06681a5660 2026-02-25T15:07:54.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f066815f9f0 con 0x7f06681a5660 2026-02-25T15:07:54.450 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0664003470 con 0x7f06681a5660 2026-02-25T15:07:54.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:54.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:54.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 10 monclient(hunting): dump: 2026-02-25T15:07:54.451 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.451 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.451 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.451 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.451 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.451 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.451 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.451 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.452 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.452 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:54.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3435413470 0 0) 0x7f066815fcd0 con 0x7f0668059e10 2026-02-25T15:07:54.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0638003610 con 0x7f0668059e10 2026-02-25T15:07:54.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0658003310 con 0x7f0668059e10 2026-02-25T15:07:54.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:54.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:54.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 10 monclient(hunting): dump: 2026-02-25T15:07:54.452 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.452 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.452 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.453 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.453 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.453 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.453 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.453 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.453 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.453 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2357596984 0 0) 0x7f066805c560 con 0x7f066815e2a0 2026-02-25T15:07:54.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f066815fcd0 con 0x7f066815e2a0 2026-02-25T15:07:54.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0650002ff0 con 0x7f066815e2a0 2026-02-25T15:07:54.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:54.453 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:54.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.437+0000 7f065d7fa640 10 monclient(hunting): dump: 2026-02-25T15:07:54.454 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.454 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.454 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.454 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.454 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.454 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.454 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.454 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.454 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.454 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3144131678 0 0) 0x7f066815f9f0 con 0x7f06681a5660 2026-02-25T15:07:54.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 >> v1:10.20.193.94:6789/0 conn(0x7f066815e2a0 legacy=0x7f06681a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 >> v1:10.20.193.93:6789/0 conn(0x7f0668059e10 legacy=0x7f066812abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f065d7fa640 10 monclient: _finish_hunting 0 2026-02-25T15:07:54.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f065d7fa640 1 monclient: found mon.a 2026-02-25T15:07:54.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f065d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:54.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f065d7fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:54.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066815ffb0 con 0x7f06681a5660 2026-02-25T15:07:54.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f065d7fa640 10 monclient: _finish_auth 0 2026-02-25T15:07:54.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f065d7fa640 10 monclient: _check_auth_tickets 2026-02-25T15:07:54.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f065d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:54.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f066f283640 5 monclient: authenticate success, global_id 30786 2026-02-25T15:07:54.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f066f283640 10 monclient: _renew_subs 2026-02-25T15:07:54.455 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f066f283640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:54.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f066f283640 1 -- 10.20.193.90:0/2499429412 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f06680571a0 con 0x7f06681a5660 2026-02-25T15:07:54.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f066f283640 10 client.30786.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:54.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f066f283640 10 monclient: _renew_subs 2026-02-25T15:07:54.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f066f283640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:54.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f066f283640 1 -- 10.20.193.90:0/2499429412 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0668160990 con 0x7f06681a5660 2026-02-25T15:07:54.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.438+0000 7f066f283640 1 librados: init done 2026-02-25T15:07:54.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.439+0000 7f066f283640 10 client.30786.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:54.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.439+0000 7f066f283640 15 client.30786.objecter _session_command_op_assign -1 1 2026-02-25T15:07:54.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.439+0000 7f066f283640 20 client.30786.objecter _get_session osd=-1 returning homeless 2026-02-25T15:07:54.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.439+0000 7f066f283640 10 client.30786.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:54.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.439+0000 7f066f283640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:54.456 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.439+0000 7f066f283640 1 -- 10.20.193.90:0/2499429412 --> v1:10.20.193.90:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f066815f7a0 con 0x7f06681a5660 2026-02-25T15:07:54.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.447+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f066815f7a0 con 0x7f06681a5660 2026-02-25T15:07:54.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.447+0000 7f065d7fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:54.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.447+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f0664003d30 con 0x7f06681a5660 2026-02-25T15:07:54.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.447+0000 7f065d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:54.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.447+0000 7f065d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:54.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.447+0000 7f065d7fa640 10 monclient: dump: 2026-02-25T15:07:54.459 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.459 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.459 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.459 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.459 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.459 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.459 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.460 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.460 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.460 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.447+0000 7f065effd640 4 set_mon_vals no callback set 2026-02-25T15:07:54.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.447+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f0664003f40 con 0x7f06681a5660 2026-02-25T15:07:54.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.448+0000 7f066cff8640 10 monclient: get_auth_request con 0x7f063807a0f0 auth_method 0 2026-02-25T15:07:54.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.448+0000 7f065d7fa640 10 client.30786.objecter ms_handle_connect 0x7f063807a0f0 2026-02-25T15:07:54.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(337..337 src has 1..337) ==== 8959+0+0 (unknown 3556021173 0 0) 0x7f0664003d30 con 0x7f06681a5660 2026-02-25T15:07:54.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 10 client.30786.objecter ms_dispatch2 0x7f0668000b90 osd_map(337..337 src has 1..337) 2026-02-25T15:07:54.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 3 client.30786.objecter handle_osd_map got epochs [337,337] > 0 2026-02-25T15:07:54.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 3 client.30786.objecter handle_osd_map decoding full epoch 337 2026-02-25T15:07:54.460 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 10 client.30786.objecter checking command 1 2026-02-25T15:07:54.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 20 client.30786.objecter _get_session s=0x7f06380a43a0 osd=0 3 2026-02-25T15:07:54.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 20 client.30786.objecter put_session s=0x7f06380a43a0 osd=0 3 2026-02-25T15:07:54.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 15 client.30786.objecter _session_command_op_remove -1 1 2026-02-25T15:07:54.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 20 client.30786.objecter _get_session s=0x7f06380a43a0 osd=0 3 2026-02-25T15:07:54.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 20 client.30786.objecter get_session s=0x7f06380a43a0 osd=0 3 2026-02-25T15:07:54.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 15 client.30786.objecter _session_command_op_assign 0 1 2026-02-25T15:07:54.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 20 client.30786.objecter put_session s=0x7f06380a43a0 osd=0 4 2026-02-25T15:07:54.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 10 client.30786.objecter _send_command 1 2026-02-25T15:07:54.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 --> v1:10.20.193.90:6808/3221464204 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f06380c6750 con 0x7f06380a4a50 2026-02-25T15:07:54.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 20 client.30786.objecter dump_active .. 0 homeless 2026-02-25T15:07:54.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_get_version_reply(handle=1 version=337) ==== 24+0+0 (unknown 676316437 0 0) 0x7f066405b240 con 0x7f06681a5660 2026-02-25T15:07:54.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065d7fa640 10 monclient: handle_get_version_reply finishing 1 version 337 2026-02-25T15:07:54.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.449+0000 7f065ffff640 10 monclient: get_auth_request con 0x7f06380a4a50 auth_method 0 2026-02-25T15:07:54.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.450+0000 7f065d7fa640 10 client.30786.objecter ms_handle_connect 0x7f06380a4a50 2026-02-25T15:07:54.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.462+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== osd.0 v1:10.20.193.90:6808/3221464204 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f06380c6750 con 0x7f06380a4a50 2026-02-25T15:07:54.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.462+0000 7f065d7fa640 10 client.30786.objecter ms_dispatch2 0x7f0668000b90 command_reply(tid 1: 0 ) 2026-02-25T15:07:54.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.462+0000 7f065d7fa640 10 client.30786.objecter _finish_command 1 = system:0 2026-02-25T15:07:54.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.462+0000 7f065d7fa640 20 client.30786.objecter put_session s=0x7f06380a43a0 osd=0 4 2026-02-25T15:07:54.464 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.462+0000 7f065d7fa640 15 client.30786.objecter _session_command_op_remove 0 1 2026-02-25T15:07:54.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.468+0000 7f066f283640 10 client.30786.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:07:54.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.468+0000 7f066f283640 15 client.30786.objecter _session_command_op_assign -1 2 2026-02-25T15:07:54.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.468+0000 7f066f283640 20 client.30786.objecter _get_session s=0x7f06380a43a0 osd=0 3 2026-02-25T15:07:54.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.468+0000 7f066f283640 20 client.30786.objecter put_session s=0x7f06380a43a0 osd=0 3 2026-02-25T15:07:54.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.468+0000 7f066f283640 20 client.30786.objecter _get_session s=0x7f06380a43a0 osd=0 3 2026-02-25T15:07:54.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.468+0000 7f066f283640 15 client.30786.objecter _session_command_op_remove -1 2 2026-02-25T15:07:54.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.468+0000 7f066f283640 20 client.30786.objecter get_session s=0x7f06380a43a0 osd=0 3 2026-02-25T15:07:54.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.468+0000 7f066f283640 15 client.30786.objecter _session_command_op_assign 0 2 2026-02-25T15:07:54.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.468+0000 7f066f283640 20 client.30786.objecter put_session s=0x7f06380a43a0 osd=0 4 2026-02-25T15:07:54.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.468+0000 7f066f283640 10 client.30786.objecter _send_command 2 2026-02-25T15:07:54.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.468+0000 7f066f283640 1 -- 10.20.193.90:0/2499429412 --> v1:10.20.193.90:6808/3221464204 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f066815ce80 con 0x7f06380a4a50 2026-02-25T15:07:54.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f065d7fa640 1 -- 10.20.193.90:0/2499429412 <== osd.0 v1:10.20.193.90:6808/3221464204 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f066815ce80 con 0x7f06380a4a50 2026-02-25T15:07:54.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f065d7fa640 10 client.30786.objecter ms_dispatch2 0x7f0668000b90 command_reply(tid 2: 0 ) 2026-02-25T15:07:54.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f065d7fa640 10 client.30786.objecter _finish_command 2 = system:0 2026-02-25T15:07:54.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f065d7fa640 20 client.30786.objecter put_session s=0x7f06380a43a0 osd=0 4 2026-02-25T15:07:54.479 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f065d7fa640 15 client.30786.objecter _session_command_op_remove 0 2 2026-02-25T15:07:54.480 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:07:54.480 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-25T15:07:54.480 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:07:54.481 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f066f283640 10 librados: watch_flush enter 2026-02-25T15:07:54.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f066f283640 10 librados: watch_flush exit 2026-02-25T15:07:54.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f066f283640 10 client.30786.objecter close_session for osd.0 2026-02-25T15:07:54.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f066f283640 1 -- 10.20.193.90:0/2499429412 >> v1:10.20.193.90:6808/3221464204 conn(0x7f06380a4a50 legacy=0x7f06380c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f066f283640 20 client.30786.objecter put_session s=0x7f06380a43a0 osd=0 1 2026-02-25T15:07:54.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f066f283640 20 client.30786.objecter shutdown clearing up homeless session... 2026-02-25T15:07:54.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f066f283640 10 client.30786.objecter successfully canceled tick 2026-02-25T15:07:54.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f066f283640 1 -- 10.20.193.90:0/2499429412 >> v1:10.20.193.189:6800/3647375982 conn(0x7f063807a0f0 legacy=0x7f063809a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f066f283640 10 monclient: shutdown 2026-02-25T15:07:54.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f066f283640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:54.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f066f283640 1 -- 10.20.193.90:0/2499429412 >> v1:10.20.193.90:6789/0 conn(0x7f06681a5660 legacy=0x7f0668056a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.482 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f066d7f9640 1 -- 10.20.193.90:0/2499429412 reap_dead start 2026-02-25T15:07:54.483 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f066f283640 1 -- 10.20.193.90:0/2499429412 shutdown_connections 2026-02-25T15:07:54.483 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.476+0000 7f066f283640 1 -- 10.20.193.90:0/2499429412 >> 10.20.193.90:0/2499429412 conn(0x7f06680b2f60 msgr2=0x7f066815d160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:54.483 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.477+0000 7f066f283640 1 -- 10.20.193.90:0/2499429412 shutdown_connections 2026-02-25T15:07:54.483 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.477+0000 7f066f283640 1 -- 10.20.193.90:0/2499429412 wait complete. 2026-02-25T15:07:54.483 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.477+0000 7f066f283640 1 librados: shutdown 2026-02-25T15:07:54.488 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-25T15:07:54.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.556+0000 7f46b1d09640 10 monclient: get_monmap_and_config 2026-02-25T15:07:54.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.556+0000 7f46b1d09640 10 monclient: build_initial_monmap 2026-02-25T15:07:54.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.556+0000 7f46b1d09640 10 monclient: monmap: 2026-02-25T15:07:54.562 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:54.563 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.563 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:54.557228+0000 2026-02-25T15:07:54.563 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:54.557228+0000 2026-02-25T15:07:54.563 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:54.563 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.563 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:54.563 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:54.563 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:54.563 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.557+0000 7f46b1d09640 1 Processor -- start 2026-02-25T15:07:54.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.557+0000 7f46b1d09640 1 -- start start 2026-02-25T15:07:54.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.557+0000 7f46b1d09640 10 monclient: init 2026-02-25T15:07:54.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.557+0000 7f46b1d09640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:54.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.557+0000 7f46b1d09640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:07:54.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.557+0000 7f46b1d09640 10 monclient(hunting): picked mon.noname-a con 0x7f46ac157780 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:54.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.557+0000 7f46b1d09640 10 monclient(hunting): picked mon.noname-c con 0x7f46ac157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:54.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.557+0000 7f46b1d09640 10 monclient(hunting): picked mon.noname-b con 0x7f46ac164d00 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:54.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.557+0000 7f46b1d09640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46ac05de70 con 0x7f46ac157780 2026-02-25T15:07:54.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.557+0000 7f46b1d09640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46ac055b20 con 0x7f46ac164d00 2026-02-25T15:07:54.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.557+0000 7f46b1d09640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46ac055e00 con 0x7f46ac157b50 2026-02-25T15:07:54.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.557+0000 7f46b1d09640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:54.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.557+0000 7f46b1d09640 10 monclient(hunting): authenticate will time out at 3364.157570s 2026-02-25T15:07:54.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.557+0000 7f46ab7fe640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f46ac157780 0x7f46ac056ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39602/0 (socket says 10.20.193.90:39602) 2026-02-25T15:07:54.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.557+0000 7f46ab7fe640 1 -- 10.20.193.90:0/3199606 learned_addr learned my addr 10.20.193.90:0/3199606 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:54.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 1 -- 10.20.193.90:0/3199606 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 665795011 0 0) 0x7f46ac05de70 con 0x7f46ac157780 2026-02-25T15:07:54.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 10 monclient(hunting): my global_id is 30789 2026-02-25T15:07:54.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 1 -- 10.20.193.90:0/3199606 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4690003610 con 0x7f46ac157780 2026-02-25T15:07:54.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 1 -- 10.20.193.90:0/3199606 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3547801403 0 0) 0x7f46ac055e00 con 0x7f46ac157b50 2026-02-25T15:07:54.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 10 monclient(hunting): my global_id is 28559 2026-02-25T15:07:54.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 1 -- 10.20.193.90:0/3199606 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46ac05de70 con 0x7f46ac157b50 2026-02-25T15:07:54.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 1 -- 10.20.193.90:0/3199606 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 130381885 0 0) 0x7f46ac055b20 con 0x7f46ac164d00 2026-02-25T15:07:54.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 10 monclient(hunting): my global_id is 27421 2026-02-25T15:07:54.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.559+0000 7f46aa7fc640 1 -- 10.20.193.90:0/3199606 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46ac055e00 con 0x7f46ac164d00 2026-02-25T15:07:54.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 1 -- 10.20.193.90:0/3199606 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1239031013 0 0) 0x7f46ac055e00 con 0x7f46ac164d00 2026-02-25T15:07:54.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 1 -- 10.20.193.90:0/3199606 >> v1:10.20.193.94:6789/0 conn(0x7f46ac157b50 legacy=0x7f46ac1645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 1 -- 10.20.193.90:0/3199606 >> v1:10.20.193.90:6789/0 conn(0x7f46ac157780 legacy=0x7f46ac056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:07:54.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 1 monclient: found mon.noname-b 2026-02-25T15:07:54.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:54.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:07:54.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 1 -- 10.20.193.90:0/3199606 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac0560e0 con 0x7f46ac164d00 2026-02-25T15:07:54.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 10 monclient: _finish_auth 0 2026-02-25T15:07:54.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:07:54.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:54.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 1 -- 10.20.193.90:0/3199606 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f46a0003150 con 0x7f46ac164d00 2026-02-25T15:07:54.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:54.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:54.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 10 monclient: dump: 2026-02-25T15:07:54.574 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.574 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.574 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.574 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.574 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.575 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.575 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.575 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.575 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.575 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46b1d09640 5 monclient: authenticate success, global_id 27421 2026-02-25T15:07:54.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46b1d09640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:54.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 1 -- 10.20.193.90:0/3199606 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f46a0003320 con 0x7f46ac164d00 2026-02-25T15:07:54.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:54.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 1 -- 10.20.193.90:0/3199606 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f46a0003760 con 0x7f46ac164d00 2026-02-25T15:07:54.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:54.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:54.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46aa7fc640 10 monclient: dump: 2026-02-25T15:07:54.576 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.576 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.576 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.576 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.576 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.576 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.576 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.576 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.576 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.576 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46b1d09640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:54.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.561+0000 7f46b1d09640 4 set_mon_vals no callback set 2026-02-25T15:07:54.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.562+0000 7f46b1d09640 10 monclient: shutdown 2026-02-25T15:07:54.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.562+0000 7f46b1d09640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:54.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.562+0000 7f46b1d09640 1 -- 10.20.193.90:0/3199606 >> v1:10.20.193.93:6789/0 conn(0x7f46ac164d00 legacy=0x7f46ac1c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.562+0000 7f46b1d09640 1 -- 10.20.193.90:0/3199606 shutdown_connections 2026-02-25T15:07:54.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.562+0000 7f46b1d09640 1 -- 10.20.193.90:0/3199606 >> 10.20.193.90:0/3199606 conn(0x7f46ac0b2ef0 msgr2=0x7f46ac0b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:54.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.562+0000 7f46b1d09640 1 -- 10.20.193.90:0/3199606 shutdown_connections 2026-02-25T15:07:54.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.562+0000 7f46b1d09640 1 -- 10.20.193.90:0/3199606 wait complete. 2026-02-25T15:07:54.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.562+0000 7f46b1d09640 10 monclient: build_initial_monmap 2026-02-25T15:07:54.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.562+0000 7f46b1d09640 10 monclient: monmap: 2026-02-25T15:07:54.577 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:54.577 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:54.578 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:54.578 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:54.578 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:54.578 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:54.578 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:54.578 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:54.578 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:54.578 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.562+0000 7f46b1d09640 1 librados: starting msgr at 2026-02-25T15:07:54.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.562+0000 7f46b1d09640 1 librados: starting objecter 2026-02-25T15:07:54.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:54.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 1 Processor -- start 2026-02-25T15:07:54.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 1 -- start start 2026-02-25T15:07:54.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 1 librados: setting wanted keys 2026-02-25T15:07:54.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 1 librados: calling monclient init 2026-02-25T15:07:54.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 10 monclient: init 2026-02-25T15:07:54.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:54.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:07:54.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 10 monclient(hunting): picked mon.noname-b con 0x7f46ac157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:54.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 10 monclient(hunting): picked mon.noname-c con 0x7f46ac157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:54.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 10 monclient(hunting): picked mon.noname-a con 0x7f46ac164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:54.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46ac134540 con 0x7f46ac164d00 2026-02-25T15:07:54.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46ac13cad0 con 0x7f46ac157780 2026-02-25T15:07:54.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46ac13e610 con 0x7f46ac157b50 2026-02-25T15:07:54.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:54.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46b1d09640 10 monclient(hunting): authenticate will time out at 3364.163481s 2026-02-25T15:07:54.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46abfff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f46ac164d00 0x7f46ac146910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39604/0 (socket says 10.20.193.90:39604) 2026-02-25T15:07:54.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f46abfff640 1 -- 10.20.193.90:0/1742288967 learned_addr learned my addr 10.20.193.90:0/1742288967 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:54.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f4687fff640 10 client.?.objecter ms_handle_connect 0x7f46ac164d00 2026-02-25T15:07:54.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.563+0000 7f4687fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:54.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.564+0000 7f4687fff640 10 client.?.objecter ms_handle_connect 0x7f46ac157780 2026-02-25T15:07:54.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.564+0000 7f4687fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:54.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.564+0000 7f4687fff640 10 client.?.objecter ms_handle_connect 0x7f46ac157b50 2026-02-25T15:07:54.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.564+0000 7f4687fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:54.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.564+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4246440718 0 0) 0x7f46ac134540 con 0x7f46ac164d00 2026-02-25T15:07:54.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.564+0000 7f4687fff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.564+0000 7f4687fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.564+0000 7f4687fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.564+0000 7f4687fff640 10 monclient(hunting): my global_id is 30792 2026-02-25T15:07:54.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.564+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f467c003610 con 0x7f46ac164d00 2026-02-25T15:07:54.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.565+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1723889877 0 0) 0x7f46ac13e610 con 0x7f46ac157b50 2026-02-25T15:07:54.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.565+0000 7f4687fff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.565+0000 7f4687fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.565+0000 7f4687fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.565+0000 7f4687fff640 10 monclient(hunting): my global_id is 28562 2026-02-25T15:07:54.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.565+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46ac134540 con 0x7f46ac157b50 2026-02-25T15:07:54.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.565+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2472045152 0 0) 0x7f46ac13cad0 con 0x7f46ac157780 2026-02-25T15:07:54.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.565+0000 7f4687fff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.582 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.565+0000 7f4687fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.582 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.565+0000 7f4687fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.582 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.565+0000 7f4687fff640 10 monclient(hunting): my global_id is 27424 2026-02-25T15:07:54.582 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.565+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46ac13e610 con 0x7f46ac157780 2026-02-25T15:07:54.582 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2771344856 0 0) 0x7f467c003610 con 0x7f46ac164d00 2026-02-25T15:07:54.582 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f46ac13cad0 con 0x7f46ac164d00 2026-02-25T15:07:54.582 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f46a0003670 con 0x7f46ac164d00 2026-02-25T15:07:54.582 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:54.582 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:54.582 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 10 monclient(hunting): dump: 2026-02-25T15:07:54.582 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.582 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.582 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.583 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.583 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.583 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.583 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.583 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.583 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.583 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.583 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:54.583 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2024891527 0 0) 0x7f46ac134540 con 0x7f46ac157b50 2026-02-25T15:07:54.583 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f467c003610 con 0x7f46ac157b50 2026-02-25T15:07:54.583 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f469c0030b0 con 0x7f46ac157b50 2026-02-25T15:07:54.583 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:54.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:54.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 10 monclient(hunting): dump: 2026-02-25T15:07:54.584 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.584 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.584 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.584 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.584 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.584 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.584 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.584 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.584 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.584 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1561368378 0 0) 0x7f46ac13e610 con 0x7f46ac157780 2026-02-25T15:07:54.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f46ac134540 con 0x7f46ac157780 2026-02-25T15:07:54.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4698003030 con 0x7f46ac157780 2026-02-25T15:07:54.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:54.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:54.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.567+0000 7f4687fff640 10 monclient(hunting): dump: 2026-02-25T15:07:54.585 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.585 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.585 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.585 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.585 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.585 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.585 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.586 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.586 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.586 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 507663008 0 0) 0x7f46ac13cad0 con 0x7f46ac164d00 2026-02-25T15:07:54.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 >> v1:10.20.193.94:6789/0 conn(0x7f46ac157b50 legacy=0x7f46ac134a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 >> v1:10.20.193.93:6789/0 conn(0x7f46ac157780 legacy=0x7f46ac13c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f4687fff640 10 monclient: _finish_hunting 0 2026-02-25T15:07:54.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f4687fff640 1 monclient: found mon.a 2026-02-25T15:07:54.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f4687fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:54.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f4687fff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:54.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac13e8f0 con 0x7f46ac164d00 2026-02-25T15:07:54.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f4687fff640 10 monclient: _finish_auth 0 2026-02-25T15:07:54.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f4687fff640 10 monclient: _check_auth_tickets 2026-02-25T15:07:54.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f4687fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:54.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f46b1d09640 5 monclient: authenticate success, global_id 30792 2026-02-25T15:07:54.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f46b1d09640 10 monclient: _renew_subs 2026-02-25T15:07:54.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f46b1d09640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:54.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f46b1d09640 1 -- 10.20.193.90:0/1742288967 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f46ac13ebd0 con 0x7f46ac164d00 2026-02-25T15:07:54.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f46b1d09640 10 client.30792.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:54.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f46b1d09640 10 monclient: _renew_subs 2026-02-25T15:07:54.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f46b1d09640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:54.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f46b1d09640 1 -- 10.20.193.90:0/1742288967 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f46ac13f190 con 0x7f46ac164d00 2026-02-25T15:07:54.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.568+0000 7f46b1d09640 1 librados: init done 2026-02-25T15:07:54.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.569+0000 7f46b1d09640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:54.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.569+0000 7f46b1d09640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:54.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.569+0000 7f46b1d09640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:54.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.569+0000 7f46b1d09640 1 -- 10.20.193.90:0/1742288967 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ac163930 con 0x7f46ac164d00 2026-02-25T15:07:54.660 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.658+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f46a0003810 con 0x7f46ac164d00 2026-02-25T15:07:54.660 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.658+0000 7f4687fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:54.660 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.658+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f46a0003c90 con 0x7f46ac164d00 2026-02-25T15:07:54.660 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.658+0000 7f4687fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:54.660 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.658+0000 7f4687fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:54.660 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.658+0000 7f4687fff640 10 monclient: dump: 2026-02-25T15:07:54.661 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.661 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.661 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.661 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.661 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.661 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.661 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.661 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.661 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.661 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.661 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.658+0000 7f46a9ffb640 4 set_mon_vals no callback set 2026-02-25T15:07:54.662 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.659+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f46a0003e70 con 0x7f46ac164d00 2026-02-25T15:07:54.662 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.659+0000 7f46ab7fe640 10 monclient: get_auth_request con 0x7f467c07a0f0 auth_method 0 2026-02-25T15:07:54.662 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.660+0000 7f4687fff640 10 client.30792.objecter ms_handle_connect 0x7f467c07a0f0 2026-02-25T15:07:54.663 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.661+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(337..337 src has 1..337) ==== 8959+0+0 (unknown 3556021173 0 0) 0x7f46a0095c30 con 0x7f46ac164d00 2026-02-25T15:07:54.663 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.661+0000 7f4687fff640 10 client.30792.objecter ms_dispatch2 0x7f46ac000b90 osd_map(337..337 src has 1..337) 2026-02-25T15:07:54.663 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.661+0000 7f4687fff640 3 client.30792.objecter handle_osd_map got epochs [337,337] > 0 2026-02-25T15:07:54.663 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.661+0000 7f4687fff640 3 client.30792.objecter handle_osd_map decoding full epoch 337 2026-02-25T15:07:54.664 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.662+0000 7f4687fff640 20 client.30792.objecter dump_active .. 0 homeless 2026-02-25T15:07:54.664 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.662+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f46ac163930 con 0x7f46ac164d00 2026-02-25T15:07:54.664 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.662+0000 7f4687fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:54.664 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.662+0000 7f4687fff640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:54.770 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.769+0000 7f46b1d09640 1 -- 10.20.193.90:0/1742288967 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f46ac165740 con 0x7f467c07a0f0 2026-02-25T15:07:54.771 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.769+0000 7f4687fff640 1 -- 10.20.193.90:0/1742288967 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (unknown 2057319476 0 0) 0x7f46ac165740 con 0x7f467c07a0f0 2026-02-25T15:07:54.771 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 0 to deep-scrub 2026-02-25T15:07:54.772 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.770+0000 7f46b1d09640 10 librados: watch_flush enter 2026-02-25T15:07:54.772 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.770+0000 7f46b1d09640 10 librados: watch_flush exit 2026-02-25T15:07:54.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.770+0000 7f46b1d09640 20 client.30792.objecter shutdown clearing up homeless session... 2026-02-25T15:07:54.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.770+0000 7f46b1d09640 10 client.30792.objecter successfully canceled tick 2026-02-25T15:07:54.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.770+0000 7f46b1d09640 1 -- 10.20.193.90:0/1742288967 >> v1:10.20.193.189:6800/3647375982 conn(0x7f467c07a0f0 legacy=0x7f467c09a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.770+0000 7f46b1d09640 10 monclient: shutdown 2026-02-25T15:07:54.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.770+0000 7f46b1d09640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:54.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.770+0000 7f46b1d09640 1 -- 10.20.193.90:0/1742288967 >> v1:10.20.193.90:6789/0 conn(0x7f46ac164d00 legacy=0x7f46ac146910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.770+0000 7f46b1d09640 1 -- 10.20.193.90:0/1742288967 shutdown_connections 2026-02-25T15:07:54.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.770+0000 7f46b1d09640 1 -- 10.20.193.90:0/1742288967 >> 10.20.193.90:0/1742288967 conn(0x7f46ac0b2ef0 msgr2=0x7f46ac05c220 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:54.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.770+0000 7f46b1d09640 1 -- 10.20.193.90:0/1742288967 shutdown_connections 2026-02-25T15:07:54.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.770+0000 7f46b1d09640 1 -- 10.20.193.90:0/1742288967 wait complete. 2026-02-25T15:07:54.773 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.770+0000 7f46b1d09640 1 librados: shutdown 2026-02-25T15:07:54.781 INFO:tasks.ceph:Scrubbing osd.4 2026-02-25T15:07:54.781 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:07:54.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.850+0000 7fd81760b640 10 monclient: get_monmap_and_config 2026-02-25T15:07:54.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.850+0000 7fd81760b640 10 monclient: build_initial_monmap 2026-02-25T15:07:54.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.850+0000 7fd81760b640 10 monclient: monmap: 2026-02-25T15:07:54.852 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:54.852 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.852 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:54.850893+0000 2026-02-25T15:07:54.853 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:54.850893+0000 2026-02-25T15:07:54.853 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:54.853 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.853 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:54.853 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:54.853 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:54.853 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.850+0000 7fd81760b640 1 Processor -- start 2026-02-25T15:07:54.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.850+0000 7fd81760b640 1 -- start start 2026-02-25T15:07:54.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.850+0000 7fd81760b640 10 monclient: init 2026-02-25T15:07:54.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.851+0000 7fd81760b640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:54.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.851+0000 7fd81760b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:07:54.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.851+0000 7fd81760b640 10 monclient(hunting): picked mon.noname-a con 0x7fd81015e2a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:54.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.851+0000 7fd81760b640 10 monclient(hunting): picked mon.noname-c con 0x7fd810059e10 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:54.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.851+0000 7fd81760b640 10 monclient(hunting): picked mon.noname-b con 0x7fd8101a5660 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:54.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.851+0000 7fd81760b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd810162df0 con 0x7fd81015e2a0 2026-02-25T15:07:54.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.851+0000 7fd81760b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8101630d0 con 0x7fd8101a5660 2026-02-25T15:07:54.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.851+0000 7fd81760b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8101633b0 con 0x7fd810059e10 2026-02-25T15:07:54.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.851+0000 7fd81760b640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:54.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.851+0000 7fd81760b640 10 monclient(hunting): authenticate will time out at 3364.451257s 2026-02-25T15:07:54.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.851+0000 7fd815b81640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fd8101a5660 0x7fd8101c5a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45476/0 (socket says 10.20.193.90:45476) 2026-02-25T15:07:54.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.851+0000 7fd815b81640 1 -- 10.20.193.90:0/61935277 learned_addr learned my addr 10.20.193.90:0/61935277 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:54.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.851+0000 7fd815380640 1 --1- 10.20.193.90:0/61935277 >> v1:10.20.193.90:6789/0 conn(0x7fd81015e2a0 0x7fd81005da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39610/0 (socket says 10.20.193.90:39610) 2026-02-25T15:07:54.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 1 -- 10.20.193.90:0/61935277 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 761796663 0 0) 0x7fd8101630d0 con 0x7fd8101a5660 2026-02-25T15:07:54.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 10 monclient(hunting): my global_id is 27427 2026-02-25T15:07:54.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 1 -- 10.20.193.90:0/61935277 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7f8003610 con 0x7fd8101a5660 2026-02-25T15:07:54.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 1 -- 10.20.193.90:0/61935277 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3033732953 0 0) 0x7fd810162df0 con 0x7fd81015e2a0 2026-02-25T15:07:54.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 10 monclient(hunting): my global_id is 30795 2026-02-25T15:07:54.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 1 -- 10.20.193.90:0/61935277 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8101630d0 con 0x7fd81015e2a0 2026-02-25T15:07:54.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 1 -- 10.20.193.90:0/61935277 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2088531351 0 0) 0x7fd8101633b0 con 0x7fd810059e10 2026-02-25T15:07:54.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 10 monclient(hunting): my global_id is 28565 2026-02-25T15:07:54.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.853+0000 7fd7fffff640 1 -- 10.20.193.90:0/61935277 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd810162df0 con 0x7fd810059e10 2026-02-25T15:07:54.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd7fffff640 1 -- 10.20.193.90:0/61935277 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 616882945 0 0) 0x7fd7f8003610 con 0x7fd8101a5660 2026-02-25T15:07:54.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd7fffff640 1 -- 10.20.193.90:0/61935277 >> v1:10.20.193.94:6789/0 conn(0x7fd810059e10 legacy=0x7fd81015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd7fffff640 1 -- 10.20.193.90:0/61935277 >> v1:10.20.193.90:6789/0 conn(0x7fd81015e2a0 legacy=0x7fd81005da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd7fffff640 10 monclient: _finish_hunting 0 2026-02-25T15:07:54.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd7fffff640 1 monclient: found mon.noname-b 2026-02-25T15:07:54.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd7fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:54.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd7fffff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:07:54.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd7fffff640 1 -- 10.20.193.90:0/61935277 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd810163690 con 0x7fd8101a5660 2026-02-25T15:07:54.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd7fffff640 10 monclient: _finish_auth 0 2026-02-25T15:07:54.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd7fffff640 10 monclient: _check_auth_tickets 2026-02-25T15:07:54.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd7fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:54.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd7fffff640 1 -- 10.20.193.90:0/61935277 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd80c003170 con 0x7fd8101a5660 2026-02-25T15:07:54.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd7fffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:54.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd7fffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:54.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd7fffff640 10 monclient: dump: 2026-02-25T15:07:54.860 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.860 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.860 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.860 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.860 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.860 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.860 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.861 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.861 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.861 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd81760b640 5 monclient: authenticate success, global_id 27427 2026-02-25T15:07:54.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.854+0000 7fd81760b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:54.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.855+0000 7fd7fffff640 1 -- 10.20.193.90:0/61935277 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd80c003340 con 0x7fd8101a5660 2026-02-25T15:07:54.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.855+0000 7fd7fffff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:54.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.855+0000 7fd7fffff640 1 -- 10.20.193.90:0/61935277 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd80c003780 con 0x7fd8101a5660 2026-02-25T15:07:54.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.855+0000 7fd81760b640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:54.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.855+0000 7fd81760b640 4 set_mon_vals no callback set 2026-02-25T15:07:54.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.855+0000 7fd7fffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:54.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.855+0000 7fd7fffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:54.868 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.855+0000 7fd7fffff640 10 monclient: dump: 2026-02-25T15:07:54.868 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.868 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.868 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.868 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.868 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.868 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.868 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.868 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.868 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.868 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.868 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.855+0000 7fd81760b640 10 monclient: shutdown 2026-02-25T15:07:54.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.855+0000 7fd81760b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:54.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.855+0000 7fd81760b640 1 -- 10.20.193.90:0/61935277 >> v1:10.20.193.93:6789/0 conn(0x7fd8101a5660 legacy=0x7fd8101c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.855+0000 7fd81760b640 1 -- 10.20.193.90:0/61935277 shutdown_connections 2026-02-25T15:07:54.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.855+0000 7fd81760b640 1 -- 10.20.193.90:0/61935277 >> 10.20.193.90:0/61935277 conn(0x7fd8100b2f60 msgr2=0x7fd8100b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:54.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.855+0000 7fd81760b640 1 -- 10.20.193.90:0/61935277 shutdown_connections 2026-02-25T15:07:54.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.855+0000 7fd81760b640 1 -- 10.20.193.90:0/61935277 wait complete. 2026-02-25T15:07:54.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 10 monclient: build_initial_monmap 2026-02-25T15:07:54.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 10 monclient: monmap: 2026-02-25T15:07:54.869 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:54.869 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:54.869 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:54.869 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:54.870 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:54.870 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:54.870 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:54.870 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:54.870 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:54.870 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 1 librados: starting msgr at 2026-02-25T15:07:54.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 1 librados: starting objecter 2026-02-25T15:07:54.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:54.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 1 Processor -- start 2026-02-25T15:07:54.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 1 -- start start 2026-02-25T15:07:54.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 1 librados: setting wanted keys 2026-02-25T15:07:54.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 1 librados: calling monclient init 2026-02-25T15:07:54.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 10 monclient: init 2026-02-25T15:07:54.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:54.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:07:54.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 10 monclient(hunting): picked mon.noname-c con 0x7fd810059e10 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:54.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 10 monclient(hunting): picked mon.noname-b con 0x7fd81015e2a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:54.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 10 monclient(hunting): picked mon.noname-a con 0x7fd8101a5660 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:54.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd81012b250 con 0x7fd8101a5660 2026-02-25T15:07:54.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd81012b530 con 0x7fd81015e2a0 2026-02-25T15:07:54.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8101941a0 con 0x7fd810059e10 2026-02-25T15:07:54.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:54.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.856+0000 7fd81760b640 10 monclient(hunting): authenticate will time out at 3364.456906s 2026-02-25T15:07:54.872 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.857+0000 7fd814b7f640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fd81015e2a0 0x7fd810161f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45478/0 (socket says 10.20.193.90:45478) 2026-02-25T15:07:54.872 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.857+0000 7fd814b7f640 1 -- 10.20.193.90:0/2383609020 learned_addr learned my addr 10.20.193.90:0/2383609020 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:54.872 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.857+0000 7fd7fdffb640 10 client.?.objecter ms_handle_connect 0x7fd81015e2a0 2026-02-25T15:07:54.872 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.857+0000 7fd7fdffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:54.872 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.857+0000 7fd7fdffb640 10 client.?.objecter ms_handle_connect 0x7fd8101a5660 2026-02-25T15:07:54.872 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.857+0000 7fd7fdffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:54.872 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.857+0000 7fd7fdffb640 10 client.?.objecter ms_handle_connect 0x7fd810059e10 2026-02-25T15:07:54.872 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.857+0000 7fd7fdffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:54.872 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3407163430 0 0) 0x7fd81012b530 con 0x7fd81015e2a0 2026-02-25T15:07:54.872 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.872 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.872 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 10 monclient(hunting): my global_id is 27430 2026-02-25T15:07:54.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7e0003610 con 0x7fd81015e2a0 2026-02-25T15:07:54.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2517108521 0 0) 0x7fd81012b250 con 0x7fd8101a5660 2026-02-25T15:07:54.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 10 monclient(hunting): my global_id is 30798 2026-02-25T15:07:54.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd81012b530 con 0x7fd8101a5660 2026-02-25T15:07:54.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1853912820 0 0) 0x7fd8101941a0 con 0x7fd810059e10 2026-02-25T15:07:54.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 10 monclient(hunting): my global_id is 28568 2026-02-25T15:07:54.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.858+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd81012b250 con 0x7fd810059e10 2026-02-25T15:07:54.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.860+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3304379587 0 0) 0x7fd7e0003610 con 0x7fd81015e2a0 2026-02-25T15:07:54.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.860+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd8101941a0 con 0x7fd81015e2a0 2026-02-25T15:07:54.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.860+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd8040030d0 con 0x7fd81015e2a0 2026-02-25T15:07:54.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.860+0000 7fd7fdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:54.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.860+0000 7fd7fdffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:54.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.860+0000 7fd7fdffb640 10 monclient(hunting): dump: 2026-02-25T15:07:54.874 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.874 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.874 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.874 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.874 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.875 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.875 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.875 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.875 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.875 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.860+0000 7fd7fdffb640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:54.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.860+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1101851972 0 0) 0x7fd81012b530 con 0x7fd8101a5660 2026-02-25T15:07:54.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.860+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd7e0003610 con 0x7fd8101a5660 2026-02-25T15:07:54.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.860+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd80c003690 con 0x7fd8101a5660 2026-02-25T15:07:54.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.860+0000 7fd7fdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:54.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.860+0000 7fd7fdffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:54.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.860+0000 7fd7fdffb640 10 monclient(hunting): dump: 2026-02-25T15:07:54.876 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.876 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.876 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.876 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.876 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.876 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.876 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.876 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.876 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.876 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.861+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3948544203 0 0) 0x7fd81012b250 con 0x7fd810059e10 2026-02-25T15:07:54.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.861+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd81012b530 con 0x7fd810059e10 2026-02-25T15:07:54.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.861+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd800003030 con 0x7fd810059e10 2026-02-25T15:07:54.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.861+0000 7fd7fdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:54.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.861+0000 7fd7fdffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:54.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.861+0000 7fd7fdffb640 10 monclient(hunting): dump: 2026-02-25T15:07:54.877 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.877 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.877 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.877 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.877 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.877 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.877 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.877 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.877 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.878 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1203430486 0 0) 0x7fd8101941a0 con 0x7fd81015e2a0 2026-02-25T15:07:54.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 >> v1:10.20.193.94:6789/0 conn(0x7fd810059e10 legacy=0x7fd810194f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 >> v1:10.20.193.90:6789/0 conn(0x7fd8101a5660 legacy=0x7fd81014cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd7fdffb640 10 monclient: _finish_hunting 0 2026-02-25T15:07:54.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd7fdffb640 1 monclient: found mon.b 2026-02-25T15:07:54.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd7fdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:54.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd7fdffb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:54.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd810194480 con 0x7fd81015e2a0 2026-02-25T15:07:54.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd7fdffb640 10 monclient: _finish_auth 0 2026-02-25T15:07:54.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd7fdffb640 10 monclient: _check_auth_tickets 2026-02-25T15:07:54.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd7fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:54.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd81760b640 5 monclient: authenticate success, global_id 27430 2026-02-25T15:07:54.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd81760b640 10 monclient: _renew_subs 2026-02-25T15:07:54.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd81760b640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:54.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd81760b640 1 -- 10.20.193.90:0/2383609020 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd810194760 con 0x7fd81015e2a0 2026-02-25T15:07:54.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd81760b640 10 client.27430.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:54.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd81760b640 10 monclient: _renew_subs 2026-02-25T15:07:54.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd81760b640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:54.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd81760b640 1 -- 10.20.193.90:0/2383609020 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd810195670 con 0x7fd81015e2a0 2026-02-25T15:07:54.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd81760b640 1 librados: init done 2026-02-25T15:07:54.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd81760b640 10 client.27430.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:54.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd81760b640 15 client.27430.objecter _session_command_op_assign -1 1 2026-02-25T15:07:54.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd81760b640 20 client.27430.objecter _get_session osd=-1 returning homeless 2026-02-25T15:07:54.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd81760b640 10 client.27430.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:54.880 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd81760b640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:54.880 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.862+0000 7fd81760b640 1 -- 10.20.193.90:0/2383609020 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fd810195900 con 0x7fd81015e2a0 2026-02-25T15:07:54.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.870+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd810195900 con 0x7fd81015e2a0 2026-02-25T15:07:54.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.870+0000 7fd7fdffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:54.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.870+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd8040039b0 con 0x7fd81015e2a0 2026-02-25T15:07:54.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.870+0000 7fd7fdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:54.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.870+0000 7fd7fdffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:54.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.870+0000 7fd7fdffb640 10 monclient: dump: 2026-02-25T15:07:54.882 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.882 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.882 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.883 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.883 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.883 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.883 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.883 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.883 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.883 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.883 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.870+0000 7fd7ff7fe640 4 set_mon_vals no callback set 2026-02-25T15:07:54.883 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.871+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fd804003bc0 con 0x7fd81015e2a0 2026-02-25T15:07:54.883 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.871+0000 7fd815380640 10 monclient: get_auth_request con 0x7fd7e007a090 auth_method 0 2026-02-25T15:07:54.883 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 10 client.27430.objecter ms_handle_connect 0x7fd7e007a090 2026-02-25T15:07:54.883 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(337..337 src has 1..337) ==== 8959+0+0 (unknown 3556021173 0 0) 0x7fd8040039b0 con 0x7fd81015e2a0 2026-02-25T15:07:54.884 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 10 client.27430.objecter ms_dispatch2 0x7fd810000b90 osd_map(337..337 src has 1..337) 2026-02-25T15:07:54.884 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 3 client.27430.objecter handle_osd_map got epochs [337,337] > 0 2026-02-25T15:07:54.884 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 3 client.27430.objecter handle_osd_map decoding full epoch 337 2026-02-25T15:07:54.884 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 10 client.27430.objecter checking command 1 2026-02-25T15:07:54.884 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 20 client.27430.objecter _get_session s=0x7fd7e00a4340 osd=4 3 2026-02-25T15:07:54.884 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 20 client.27430.objecter put_session s=0x7fd7e00a4340 osd=4 3 2026-02-25T15:07:54.884 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 15 client.27430.objecter _session_command_op_remove -1 1 2026-02-25T15:07:54.884 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 20 client.27430.objecter _get_session s=0x7fd7e00a4340 osd=4 3 2026-02-25T15:07:54.884 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 20 client.27430.objecter get_session s=0x7fd7e00a4340 osd=4 3 2026-02-25T15:07:54.884 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 15 client.27430.objecter _session_command_op_assign 4 1 2026-02-25T15:07:54.884 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 20 client.27430.objecter put_session s=0x7fd7e00a4340 osd=4 4 2026-02-25T15:07:54.884 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 10 client.27430.objecter _send_command 1 2026-02-25T15:07:54.885 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 --> v1:10.20.193.90:6800/1446526134 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd7e00c6650 con 0x7fd7e00a49f0 2026-02-25T15:07:54.885 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd7fdffb640 20 client.27430.objecter dump_active .. 0 homeless 2026-02-25T15:07:54.885 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.872+0000 7fd815b81640 10 monclient: get_auth_request con 0x7fd7e00a49f0 auth_method 0 2026-02-25T15:07:54.885 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.873+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_get_version_reply(handle=1 version=337) ==== 24+0+0 (unknown 676316437 0 0) 0x7fd80405aef0 con 0x7fd81015e2a0 2026-02-25T15:07:54.885 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.873+0000 7fd7fdffb640 10 monclient: handle_get_version_reply finishing 1 version 337 2026-02-25T15:07:54.885 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.874+0000 7fd7fdffb640 10 client.27430.objecter ms_handle_connect 0x7fd7e00a49f0 2026-02-25T15:07:54.886 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.883+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== osd.4 v1:10.20.193.90:6800/1446526134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7fd7e00c6650 con 0x7fd7e00a49f0 2026-02-25T15:07:54.886 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.883+0000 7fd7fdffb640 10 client.27430.objecter ms_dispatch2 0x7fd810000b90 command_reply(tid 1: 0 ) 2026-02-25T15:07:54.886 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.883+0000 7fd7fdffb640 10 client.27430.objecter _finish_command 1 = system:0 2026-02-25T15:07:54.886 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.883+0000 7fd7fdffb640 20 client.27430.objecter put_session s=0x7fd7e00a4340 osd=4 4 2026-02-25T15:07:54.886 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.883+0000 7fd7fdffb640 15 client.27430.objecter _session_command_op_remove 4 1 2026-02-25T15:07:54.892 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.890+0000 7fd81760b640 10 client.27430.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:07:54.892 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.890+0000 7fd81760b640 15 client.27430.objecter _session_command_op_assign -1 2 2026-02-25T15:07:54.892 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.890+0000 7fd81760b640 20 client.27430.objecter _get_session s=0x7fd7e00a4340 osd=4 3 2026-02-25T15:07:54.892 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.890+0000 7fd81760b640 20 client.27430.objecter put_session s=0x7fd7e00a4340 osd=4 3 2026-02-25T15:07:54.892 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.890+0000 7fd81760b640 20 client.27430.objecter _get_session s=0x7fd7e00a4340 osd=4 3 2026-02-25T15:07:54.892 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.890+0000 7fd81760b640 15 client.27430.objecter _session_command_op_remove -1 2 2026-02-25T15:07:54.892 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.890+0000 7fd81760b640 20 client.27430.objecter get_session s=0x7fd7e00a4340 osd=4 3 2026-02-25T15:07:54.892 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.890+0000 7fd81760b640 15 client.27430.objecter _session_command_op_assign 4 2 2026-02-25T15:07:54.892 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.890+0000 7fd81760b640 20 client.27430.objecter put_session s=0x7fd7e00a4340 osd=4 4 2026-02-25T15:07:54.893 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.890+0000 7fd81760b640 10 client.27430.objecter _send_command 2 2026-02-25T15:07:54.893 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.890+0000 7fd81760b640 1 -- 10.20.193.90:0/2383609020 --> v1:10.20.193.90:6800/1446526134 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd81015c270 con 0x7fd7e00a49f0 2026-02-25T15:07:54.899 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.897+0000 7fd7fdffb640 1 -- 10.20.193.90:0/2383609020 <== osd.4 v1:10.20.193.90:6800/1446526134 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7fd81015c270 con 0x7fd7e00a49f0 2026-02-25T15:07:54.899 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.897+0000 7fd7fdffb640 10 client.27430.objecter ms_dispatch2 0x7fd810000b90 command_reply(tid 2: 0 ) 2026-02-25T15:07:54.899 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.897+0000 7fd7fdffb640 10 client.27430.objecter _finish_command 2 = system:0 2026-02-25T15:07:54.899 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.897+0000 7fd7fdffb640 20 client.27430.objecter put_session s=0x7fd7e00a4340 osd=4 4 2026-02-25T15:07:54.899 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.897+0000 7fd7fdffb640 15 client.27430.objecter _session_command_op_remove 4 2 2026-02-25T15:07:54.900 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:07:54.900 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-25T15:07:54.900 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:07:54.901 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 10 librados: watch_flush enter 2026-02-25T15:07:54.901 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 10 librados: watch_flush exit 2026-02-25T15:07:54.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 10 client.27430.objecter close_session for osd.4 2026-02-25T15:07:54.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 1 -- 10.20.193.90:0/2383609020 >> v1:10.20.193.90:6800/1446526134 conn(0x7fd7e00a49f0 legacy=0x7fd7e00c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 20 client.27430.objecter put_session s=0x7fd7e00a4340 osd=4 1 2026-02-25T15:07:54.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 20 client.27430.objecter shutdown clearing up homeless session... 2026-02-25T15:07:54.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 10 client.27430.objecter successfully canceled tick 2026-02-25T15:07:54.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 1 -- 10.20.193.90:0/2383609020 >> v1:10.20.193.189:6800/3647375982 conn(0x7fd7e007a090 legacy=0x7fd7e009a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 10 monclient: shutdown 2026-02-25T15:07:54.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:54.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 1 -- 10.20.193.90:0/2383609020 >> v1:10.20.193.93:6789/0 conn(0x7fd81015e2a0 legacy=0x7fd810161f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd815b81640 1 -- 10.20.193.90:0/2383609020 reap_dead start 2026-02-25T15:07:54.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 1 -- 10.20.193.90:0/2383609020 shutdown_connections 2026-02-25T15:07:54.902 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 1 -- 10.20.193.90:0/2383609020 >> 10.20.193.90:0/2383609020 conn(0x7fd8100b2f60 msgr2=0x7fd81015c790 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:54.903 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 1 -- 10.20.193.90:0/2383609020 shutdown_connections 2026-02-25T15:07:54.903 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 1 -- 10.20.193.90:0/2383609020 wait complete. 2026-02-25T15:07:54.903 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.898+0000 7fd81760b640 1 librados: shutdown 2026-02-25T15:07:54.907 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2026-02-25T15:07:54.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.972+0000 7f52fec93640 10 monclient: get_monmap_and_config 2026-02-25T15:07:54.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.972+0000 7f52fec93640 10 monclient: build_initial_monmap 2026-02-25T15:07:54.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.972+0000 7f52fec93640 10 monclient: monmap: 2026-02-25T15:07:54.974 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:54.974 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.974 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:54.973229+0000 2026-02-25T15:07:54.974 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:54.973229+0000 2026-02-25T15:07:54.974 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:54.975 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.975 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:54.975 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:54.975 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:54.975 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.973+0000 7f52fec93640 1 Processor -- start 2026-02-25T15:07:54.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.973+0000 7f52fec93640 1 -- start start 2026-02-25T15:07:54.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.973+0000 7f52fec93640 10 monclient: init 2026-02-25T15:07:54.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.973+0000 7f52fec93640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:54.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.973+0000 7f52fec93640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:07:54.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.973+0000 7f52fec93640 10 monclient(hunting): picked mon.noname-a con 0x7f52f8059b90 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:54.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.973+0000 7f52fec93640 10 monclient(hunting): picked mon.noname-b con 0x7f52f80b27b0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:54.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.973+0000 7f52fec93640 10 monclient(hunting): picked mon.noname-c con 0x7f52f8159b60 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:54.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.973+0000 7f52fec93640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52f80b1750 con 0x7f52f8059b90 2026-02-25T15:07:54.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.973+0000 7f52fec93640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52f80b19f0 con 0x7f52f80b27b0 2026-02-25T15:07:54.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.973+0000 7f52fec93640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52f80b1cd0 con 0x7f52f8159b60 2026-02-25T15:07:54.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.973+0000 7f52fec93640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:54.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.973+0000 7f52fec93640 10 monclient(hunting): authenticate will time out at 3364.573749s 2026-02-25T15:07:54.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.974+0000 7f52fca08640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f52f8059b90 0x7f52f8057620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39638/0 (socket says 10.20.193.90:39638) 2026-02-25T15:07:54.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.974+0000 7f52fca08640 1 -- 10.20.193.90:0/3231593627 learned_addr learned my addr 10.20.193.90:0/3231593627 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:54.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 1 -- 10.20.193.90:0/3231593627 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 642893196 0 0) 0x7f52f80b1cd0 con 0x7f52f8159b60 2026-02-25T15:07:54.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 10 monclient(hunting): my global_id is 28571 2026-02-25T15:07:54.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 1 -- 10.20.193.90:0/3231593627 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52e4003610 con 0x7f52f8159b60 2026-02-25T15:07:54.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 1 -- 10.20.193.90:0/3231593627 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2573978167 0 0) 0x7f52f80b19f0 con 0x7f52f80b27b0 2026-02-25T15:07:54.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 10 monclient(hunting): my global_id is 27433 2026-02-25T15:07:54.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 1 -- 10.20.193.90:0/3231593627 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52f80b1cd0 con 0x7f52f80b27b0 2026-02-25T15:07:54.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 1 -- 10.20.193.90:0/3231593627 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2499148909 0 0) 0x7f52f80b1750 con 0x7f52f8059b90 2026-02-25T15:07:54.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 10 monclient(hunting): my global_id is 30801 2026-02-25T15:07:54.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.975+0000 7f52ef7fe640 1 -- 10.20.193.90:0/3231593627 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52f80b19f0 con 0x7f52f8059b90 2026-02-25T15:07:54.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.976+0000 7f52ef7fe640 1 -- 10.20.193.90:0/3231593627 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2747001840 0 0) 0x7f52e4003610 con 0x7f52f8159b60 2026-02-25T15:07:54.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.976+0000 7f52ef7fe640 1 -- 10.20.193.90:0/3231593627 >> v1:10.20.193.93:6789/0 conn(0x7f52f80b27b0 legacy=0x7f52f8159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.976+0000 7f52ef7fe640 1 -- 10.20.193.90:0/3231593627 >> v1:10.20.193.90:6789/0 conn(0x7f52f8059b90 legacy=0x7f52f8057620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.976+0000 7f52ef7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:07:54.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.976+0000 7f52ef7fe640 1 monclient: found mon.noname-c 2026-02-25T15:07:54.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.976+0000 7f52ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:54.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.976+0000 7f52ef7fe640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:07:54.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.976+0000 7f52ef7fe640 1 -- 10.20.193.90:0/3231593627 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f80b1fb0 con 0x7f52f8159b60 2026-02-25T15:07:54.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.976+0000 7f52ef7fe640 10 monclient: _finish_auth 0 2026-02-25T15:07:54.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.976+0000 7f52ef7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:07:54.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.976+0000 7f52ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:54.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.976+0000 7f52ef7fe640 1 -- 10.20.193.90:0/3231593627 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f52f4003170 con 0x7f52f8159b60 2026-02-25T15:07:54.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.976+0000 7f52ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:54.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.976+0000 7f52ef7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:07:54.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.976+0000 7f52ef7fe640 10 monclient: dump: 2026-02-25T15:07:54.988 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.988 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.988 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.989 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.989 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.989 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.989 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.989 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.989 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.989 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.989 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.977+0000 7f52fec93640 5 monclient: authenticate success, global_id 28571 2026-02-25T15:07:54.989 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.977+0000 7f52fec93640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:54.989 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.977+0000 7f52ef7fe640 1 -- 10.20.193.90:0/3231593627 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f52f4003340 con 0x7f52f8159b60 2026-02-25T15:07:54.989 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.977+0000 7f52ef7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:54.989 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.977+0000 7f52ef7fe640 1 -- 10.20.193.90:0/3231593627 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f52f4003780 con 0x7f52f8159b60 2026-02-25T15:07:54.990 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.977+0000 7f52ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:54.990 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.977+0000 7f52ef7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:54.990 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.977+0000 7f52ef7fe640 10 monclient: dump: 2026-02-25T15:07:54.990 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.990 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.990 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.990 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.990 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.990 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.990 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.990 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.990 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.990 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.977+0000 7f52fec93640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:54.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.977+0000 7f52fec93640 4 set_mon_vals no callback set 2026-02-25T15:07:54.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.978+0000 7f52fec93640 10 monclient: shutdown 2026-02-25T15:07:54.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.978+0000 7f52fec93640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:54.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.978+0000 7f52fec93640 1 -- 10.20.193.90:0/3231593627 >> v1:10.20.193.94:6789/0 conn(0x7f52f8159b60 legacy=0x7f52f81c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:54.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.978+0000 7f52fec93640 1 -- 10.20.193.90:0/3231593627 shutdown_connections 2026-02-25T15:07:54.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.978+0000 7f52fec93640 1 -- 10.20.193.90:0/3231593627 >> 10.20.193.90:0/3231593627 conn(0x7f52f80b7120 msgr2=0x7f52f80b7520 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:54.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.978+0000 7f52fec93640 1 -- 10.20.193.90:0/3231593627 shutdown_connections 2026-02-25T15:07:54.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.978+0000 7f52fec93640 1 -- 10.20.193.90:0/3231593627 wait complete. 2026-02-25T15:07:54.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.978+0000 7f52fec93640 10 monclient: build_initial_monmap 2026-02-25T15:07:54.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.978+0000 7f52fec93640 10 monclient: monmap: 2026-02-25T15:07:54.992 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:54.992 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:54.992 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:54.992 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:54.992 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:54.992 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:54.992 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:54.992 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:54.992 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:54.992 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.992 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.978+0000 7f52fec93640 1 librados: starting msgr at 2026-02-25T15:07:54.992 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.978+0000 7f52fec93640 1 librados: starting objecter 2026-02-25T15:07:54.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.978+0000 7f52fec93640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:54.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.978+0000 7f52fec93640 1 Processor -- start 2026-02-25T15:07:54.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fec93640 1 -- start start 2026-02-25T15:07:54.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fec93640 1 librados: setting wanted keys 2026-02-25T15:07:54.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fec93640 1 librados: calling monclient init 2026-02-25T15:07:54.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fec93640 10 monclient: init 2026-02-25T15:07:54.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fec93640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:54.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fec93640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:07:54.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fec93640 10 monclient(hunting): picked mon.noname-b con 0x7f52f8059b90 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:54.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fec93640 10 monclient(hunting): picked mon.noname-c con 0x7f52f80b27b0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:54.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fec93640 10 monclient(hunting): picked mon.noname-a con 0x7f52f8159b60 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:54.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fec93640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52f80b5700 con 0x7f52f8159b60 2026-02-25T15:07:54.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fec93640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52f8132090 con 0x7f52f8059b90 2026-02-25T15:07:54.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fec93640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52f8132370 con 0x7f52f80b27b0 2026-02-25T15:07:54.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fec93640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:54.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fec93640 10 monclient(hunting): authenticate will time out at 3364.579394s 2026-02-25T15:07:54.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fd209640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f52f8159b60 0x7f52f812fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39652/0 (socket says 10.20.193.90:39652) 2026-02-25T15:07:54.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52fd209640 1 -- 10.20.193.90:0/952438575 learned_addr learned my addr 10.20.193.90:0/952438575 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:54.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52ed7fa640 10 client.?.objecter ms_handle_connect 0x7f52f8159b60 2026-02-25T15:07:54.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52ed7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:54.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52ed7fa640 10 client.?.objecter ms_handle_connect 0x7f52f80b27b0 2026-02-25T15:07:54.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.979+0000 7f52ed7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:54.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.980+0000 7f52ed7fa640 10 client.?.objecter ms_handle_connect 0x7f52f8059b90 2026-02-25T15:07:54.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.980+0000 7f52ed7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:54.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.980+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1699297976 0 0) 0x7f52f80b5700 con 0x7f52f8159b60 2026-02-25T15:07:54.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.980+0000 7f52ed7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.980+0000 7f52ed7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.980+0000 7f52ed7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.980+0000 7f52ed7fa640 10 monclient(hunting): my global_id is 30804 2026-02-25T15:07:54.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.980+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52c8003610 con 0x7f52f8159b60 2026-02-25T15:07:54.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.980+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1643235573 0 0) 0x7f52f8132370 con 0x7f52f80b27b0 2026-02-25T15:07:54.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.980+0000 7f52ed7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.980+0000 7f52ed7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.980+0000 7f52ed7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.980+0000 7f52ed7fa640 10 monclient(hunting): my global_id is 28574 2026-02-25T15:07:54.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.980+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52f80b5700 con 0x7f52f80b27b0 2026-02-25T15:07:54.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.981+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4226984457 0 0) 0x7f52f8132090 con 0x7f52f8059b90 2026-02-25T15:07:54.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.981+0000 7f52ed7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:07:54.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.981+0000 7f52ed7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:54.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.981+0000 7f52ed7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:54.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.981+0000 7f52ed7fa640 10 monclient(hunting): my global_id is 27436 2026-02-25T15:07:54.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.981+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52f8132370 con 0x7f52f8059b90 2026-02-25T15:07:54.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4173864204 0 0) 0x7f52f80b5700 con 0x7f52f80b27b0 2026-02-25T15:07:54.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f52f8132090 con 0x7f52f80b27b0 2026-02-25T15:07:54.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 963030918 0 0) 0x7f52c8003610 con 0x7f52f8159b60 2026-02-25T15:07:54.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f52f80b5700 con 0x7f52f8159b60 2026-02-25T15:07:54.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f52f4003690 con 0x7f52f8159b60 2026-02-25T15:07:54.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:54.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:54.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 10 monclient(hunting): dump: 2026-02-25T15:07:54.997 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.997 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.997 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.997 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.997 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.997 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.997 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.997 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.997 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.997 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:54.998 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f52e00031f0 con 0x7f52f80b27b0 2026-02-25T15:07:54.998 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:54.998 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:54.998 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 10 monclient(hunting): dump: 2026-02-25T15:07:54.998 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.998 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.998 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.998 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.998 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:54.998 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:54.998 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:54.998 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:54.998 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:54.999 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:54.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4268938700 0 0) 0x7f52f8132370 con 0x7f52f8059b90 2026-02-25T15:07:54.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f52c8003610 con 0x7f52f8059b90 2026-02-25T15:07:54.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f52e8003060 con 0x7f52f8059b90 2026-02-25T15:07:54.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:54.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:54.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.983+0000 7f52ed7fa640 10 monclient(hunting): dump: 2026-02-25T15:07:54.999 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:54.999 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:54.999 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.999 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:54.999 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.000 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.000 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.000 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.000 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.000 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3266666125 0 0) 0x7f52f80b5700 con 0x7f52f8159b60 2026-02-25T15:07:55.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 >> v1:10.20.193.94:6789/0 conn(0x7f52f80b27b0 legacy=0x7f52f80b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 >> v1:10.20.193.93:6789/0 conn(0x7f52f8059b90 legacy=0x7f52f80b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52ed7fa640 10 monclient: _finish_hunting 0 2026-02-25T15:07:55.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52ed7fa640 1 monclient: found mon.a 2026-02-25T15:07:55.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52ed7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:55.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52ed7fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:55.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f8132650 con 0x7f52f8159b60 2026-02-25T15:07:55.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52ed7fa640 10 monclient: _finish_auth 0 2026-02-25T15:07:55.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52ed7fa640 10 monclient: _check_auth_tickets 2026-02-25T15:07:55.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52ed7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:55.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52fec93640 5 monclient: authenticate success, global_id 30804 2026-02-25T15:07:55.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52fec93640 10 monclient: _renew_subs 2026-02-25T15:07:55.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52fec93640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:55.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52fec93640 1 -- 10.20.193.90:0/952438575 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f52f80b6900 con 0x7f52f8159b60 2026-02-25T15:07:55.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52fec93640 10 client.30804.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:55.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52fec93640 10 monclient: _renew_subs 2026-02-25T15:07:55.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52fec93640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:55.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52fec93640 1 -- 10.20.193.90:0/952438575 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f52f812af60 con 0x7f52f8159b60 2026-02-25T15:07:55.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.984+0000 7f52fec93640 1 librados: init done 2026-02-25T15:07:55.002 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.985+0000 7f52fec93640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:55.002 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.985+0000 7f52fec93640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:55.002 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.985+0000 7f52fec93640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:55.002 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:54.985+0000 7f52fec93640 1 -- 10.20.193.90:0/952438575 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f80b1160 con 0x7f52f8159b60 2026-02-25T15:07:55.008 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.006+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f52f40032f0 con 0x7f52f8159b60 2026-02-25T15:07:55.008 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.006+0000 7f52ed7fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:55.008 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.006+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f52f4003c10 con 0x7f52f8159b60 2026-02-25T15:07:55.008 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.006+0000 7f52ed7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:55.008 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.006+0000 7f52ed7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:55.008 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.006+0000 7f52ed7fa640 10 monclient: dump: 2026-02-25T15:07:55.008 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.008 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.008 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.008 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.008 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.008 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.009 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.009 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.009 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.009 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.009 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.006+0000 7f52eeffd640 4 set_mon_vals no callback set 2026-02-25T15:07:55.010 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.007+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f52f4003df0 con 0x7f52f8159b60 2026-02-25T15:07:55.010 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.007+0000 7f52fca08640 10 monclient: get_auth_request con 0x7f52c807a040 auth_method 0 2026-02-25T15:07:55.010 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.007+0000 7f52ed7fa640 10 client.30804.objecter ms_handle_connect 0x7f52c807a040 2026-02-25T15:07:55.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.078+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(337..337 src has 1..337) ==== 8959+0+0 (unknown 3556021173 0 0) 0x7f52f4095c00 con 0x7f52f8159b60 2026-02-25T15:07:55.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.078+0000 7f52ed7fa640 10 client.30804.objecter ms_dispatch2 0x7f52f8000b90 osd_map(337..337 src has 1..337) 2026-02-25T15:07:55.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.078+0000 7f52ed7fa640 3 client.30804.objecter handle_osd_map got epochs [337,337] > 0 2026-02-25T15:07:55.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.078+0000 7f52ed7fa640 3 client.30804.objecter handle_osd_map decoding full epoch 337 2026-02-25T15:07:55.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.079+0000 7f52ed7fa640 20 client.30804.objecter dump_active .. 0 homeless 2026-02-25T15:07:55.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.079+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f52f405f560 con 0x7f52f8159b60 2026-02-25T15:07:55.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.079+0000 7f52ed7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:55.080 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.079+0000 7f52ed7fa640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:55.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.187+0000 7f52fec93640 1 -- 10.20.193.90:0/952438575 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f52f805dbf0 con 0x7f52c807a040 2026-02-25T15:07:55.189 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.187+0000 7f52ed7fa640 1 -- 10.20.193.90:0/952438575 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (unknown 3782719169 0 0) 0x7f52f805dbf0 con 0x7f52c807a040 2026-02-25T15:07:55.189 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 4 to deep-scrub 2026-02-25T15:07:55.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.189+0000 7f52fec93640 10 librados: watch_flush enter 2026-02-25T15:07:55.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.189+0000 7f52fec93640 10 librados: watch_flush exit 2026-02-25T15:07:55.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.189+0000 7f52fec93640 20 client.30804.objecter shutdown clearing up homeless session... 2026-02-25T15:07:55.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.189+0000 7f52fec93640 10 client.30804.objecter successfully canceled tick 2026-02-25T15:07:55.191 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.189+0000 7f52fec93640 1 -- 10.20.193.90:0/952438575 >> v1:10.20.193.189:6800/3647375982 conn(0x7f52c807a040 legacy=0x7f52c809a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.189+0000 7f52fec93640 10 monclient: shutdown 2026-02-25T15:07:55.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.189+0000 7f52fec93640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:55.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.189+0000 7f52fec93640 1 -- 10.20.193.90:0/952438575 >> v1:10.20.193.90:6789/0 conn(0x7f52f8159b60 legacy=0x7f52f812fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.189+0000 7f52fec93640 1 -- 10.20.193.90:0/952438575 shutdown_connections 2026-02-25T15:07:55.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.189+0000 7f52fec93640 1 -- 10.20.193.90:0/952438575 >> 10.20.193.90:0/952438575 conn(0x7f52f80b7120 msgr2=0x7f52f81315a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:55.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.189+0000 7f52fec93640 1 -- 10.20.193.90:0/952438575 shutdown_connections 2026-02-25T15:07:55.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.189+0000 7f52fec93640 1 -- 10.20.193.90:0/952438575 wait complete. 2026-02-25T15:07:55.192 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.189+0000 7f52fec93640 1 librados: shutdown 2026-02-25T15:07:55.200 INFO:tasks.ceph:Scrubbing osd.8 2026-02-25T15:07:55.200 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:07:55.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.266+0000 7f8073855640 10 monclient: get_monmap_and_config 2026-02-25T15:07:55.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.266+0000 7f8073855640 10 monclient: build_initial_monmap 2026-02-25T15:07:55.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.266+0000 7f8073855640 10 monclient: monmap: 2026-02-25T15:07:55.268 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:55.268 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.268 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:55.267635+0000 2026-02-25T15:07:55.268 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:55.267635+0000 2026-02-25T15:07:55.268 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:55.268 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.268 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:55.268 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:55.268 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:55.268 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.267+0000 7f8073855640 1 Processor -- start 2026-02-25T15:07:55.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.267+0000 7f8073855640 1 -- start start 2026-02-25T15:07:55.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.267+0000 7f8073855640 10 monclient: init 2026-02-25T15:07:55.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.267+0000 7f8073855640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:55.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.267+0000 7f8073855640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:07:55.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.267+0000 7f8073855640 10 monclient(hunting): picked mon.noname-c con 0x7f806c15e2a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:55.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.267+0000 7f8073855640 10 monclient(hunting): picked mon.noname-a con 0x7f806c059e10 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:55.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.267+0000 7f8073855640 10 monclient(hunting): picked mon.noname-b con 0x7f806c1a5660 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:55.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.267+0000 7f8073855640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f806c162df0 con 0x7f806c059e10 2026-02-25T15:07:55.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.267+0000 7f8073855640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f806c1630d0 con 0x7f806c1a5660 2026-02-25T15:07:55.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.267+0000 7f8073855640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f806c1633b0 con 0x7f806c15e2a0 2026-02-25T15:07:55.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.267+0000 7f8073855640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:55.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.267+0000 7f8073855640 10 monclient(hunting): authenticate will time out at 3364.867999s 2026-02-25T15:07:55.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.268+0000 7f8071dcb640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f806c1a5660 0x7f806c1c5a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45508/0 (socket says 10.20.193.90:45508) 2026-02-25T15:07:55.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.268+0000 7f8071dcb640 1 -- 10.20.193.90:0/311492728 learned_addr learned my addr 10.20.193.90:0/311492728 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:55.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.268+0000 7f8070dc9640 1 --1- 10.20.193.90:0/311492728 >> v1:10.20.193.90:6789/0 conn(0x7f806c059e10 0x7f806c15ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39668/0 (socket says 10.20.193.90:39668) 2026-02-25T15:07:55.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 1 -- 10.20.193.90:0/311492728 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3597326488 0 0) 0x7f806c1630d0 con 0x7f806c1a5660 2026-02-25T15:07:55.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 10 monclient(hunting): my global_id is 27439 2026-02-25T15:07:55.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 1 -- 10.20.193.90:0/311492728 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8054003610 con 0x7f806c1a5660 2026-02-25T15:07:55.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 1 -- 10.20.193.90:0/311492728 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2765308969 0 0) 0x7f806c162df0 con 0x7f806c059e10 2026-02-25T15:07:55.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 10 monclient(hunting): my global_id is 30807 2026-02-25T15:07:55.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 1 -- 10.20.193.90:0/311492728 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f806c1630d0 con 0x7f806c059e10 2026-02-25T15:07:55.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 1 -- 10.20.193.90:0/311492728 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 245026914 0 0) 0x7f806c1633b0 con 0x7f806c15e2a0 2026-02-25T15:07:55.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 10 monclient(hunting): my global_id is 28577 2026-02-25T15:07:55.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.269+0000 7f805bfff640 1 -- 10.20.193.90:0/311492728 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f806c162df0 con 0x7f806c15e2a0 2026-02-25T15:07:55.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f805bfff640 1 -- 10.20.193.90:0/311492728 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2267935871 0 0) 0x7f8054003610 con 0x7f806c1a5660 2026-02-25T15:07:55.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f805bfff640 1 -- 10.20.193.90:0/311492728 >> v1:10.20.193.94:6789/0 conn(0x7f806c15e2a0 legacy=0x7f806c05da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f805bfff640 1 -- 10.20.193.90:0/311492728 >> v1:10.20.193.90:6789/0 conn(0x7f806c059e10 legacy=0x7f806c15ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f805bfff640 10 monclient: _finish_hunting 0 2026-02-25T15:07:55.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f805bfff640 1 monclient: found mon.noname-b 2026-02-25T15:07:55.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f805bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:55.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f805bfff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:07:55.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f805bfff640 1 -- 10.20.193.90:0/311492728 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806c163690 con 0x7f806c1a5660 2026-02-25T15:07:55.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f805bfff640 10 monclient: _finish_auth 0 2026-02-25T15:07:55.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f805bfff640 10 monclient: _check_auth_tickets 2026-02-25T15:07:55.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f805bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:55.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f805bfff640 1 -- 10.20.193.90:0/311492728 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8068003170 con 0x7f806c1a5660 2026-02-25T15:07:55.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f8073855640 5 monclient: authenticate success, global_id 27439 2026-02-25T15:07:55.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f805bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:55.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f805bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:55.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f805bfff640 10 monclient: dump: 2026-02-25T15:07:55.279 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.279 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.279 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.279 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.279 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.279 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.280 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.280 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.280 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.280 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.271+0000 7f8073855640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:55.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.272+0000 7f805bfff640 1 -- 10.20.193.90:0/311492728 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8068003340 con 0x7f806c1a5660 2026-02-25T15:07:55.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.272+0000 7f805bfff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:55.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.272+0000 7f805bfff640 1 -- 10.20.193.90:0/311492728 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8068003780 con 0x7f806c1a5660 2026-02-25T15:07:55.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.272+0000 7f805bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:55.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.272+0000 7f805bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:55.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.272+0000 7f805bfff640 10 monclient: dump: 2026-02-25T15:07:55.280 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.281 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.281 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.281 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.281 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.281 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.281 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.281 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.281 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.281 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.272+0000 7f8073855640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:55.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.272+0000 7f8073855640 4 set_mon_vals no callback set 2026-02-25T15:07:55.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.272+0000 7f8073855640 10 monclient: shutdown 2026-02-25T15:07:55.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.272+0000 7f8073855640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:55.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.272+0000 7f8073855640 1 -- 10.20.193.90:0/311492728 >> v1:10.20.193.93:6789/0 conn(0x7f806c1a5660 legacy=0x7f806c1c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.272+0000 7f8073855640 1 -- 10.20.193.90:0/311492728 shutdown_connections 2026-02-25T15:07:55.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.272+0000 7f8073855640 1 -- 10.20.193.90:0/311492728 >> 10.20.193.90:0/311492728 conn(0x7f806c0b2f60 msgr2=0x7f806c0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:55.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.272+0000 7f8073855640 1 -- 10.20.193.90:0/311492728 shutdown_connections 2026-02-25T15:07:55.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.272+0000 7f8073855640 1 -- 10.20.193.90:0/311492728 wait complete. 2026-02-25T15:07:55.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 10 monclient: build_initial_monmap 2026-02-25T15:07:55.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 10 monclient: monmap: 2026-02-25T15:07:55.282 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:55.282 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:55.282 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:55.282 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:55.283 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:55.283 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:55.283 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:55.283 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:55.283 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:55.283 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 1 librados: starting msgr at 2026-02-25T15:07:55.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 1 librados: starting objecter 2026-02-25T15:07:55.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:55.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 1 Processor -- start 2026-02-25T15:07:55.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 1 -- start start 2026-02-25T15:07:55.283 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 1 librados: setting wanted keys 2026-02-25T15:07:55.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 1 librados: calling monclient init 2026-02-25T15:07:55.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 10 monclient: init 2026-02-25T15:07:55.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:55.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:07:55.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 10 monclient(hunting): picked mon.noname-b con 0x7f806c059e10 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:55.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 10 monclient(hunting): picked mon.noname-a con 0x7f806c15e2a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:55.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 10 monclient(hunting): picked mon.noname-c con 0x7f806c1a5660 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:55.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f806c12a760 con 0x7f806c15e2a0 2026-02-25T15:07:55.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f806c199570 con 0x7f806c059e10 2026-02-25T15:07:55.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f806c12a9e0 con 0x7f806c1a5660 2026-02-25T15:07:55.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:55.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.273+0000 7f8073855640 10 monclient(hunting): authenticate will time out at 3364.873937s 2026-02-25T15:07:55.284 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.274+0000 7f80715ca640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f806c059e10 0x7f806c05c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45510/0 (socket says 10.20.193.90:45510) 2026-02-25T15:07:55.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.274+0000 7f80715ca640 1 -- 10.20.193.90:0/3957528640 learned_addr learned my addr 10.20.193.90:0/3957528640 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:55.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.274+0000 7f8059ffb640 10 client.?.objecter ms_handle_connect 0x7f806c059e10 2026-02-25T15:07:55.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.274+0000 7f8059ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:55.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.274+0000 7f8059ffb640 10 client.?.objecter ms_handle_connect 0x7f806c1a5660 2026-02-25T15:07:55.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.274+0000 7f8059ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:55.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.274+0000 7f8059ffb640 10 client.?.objecter ms_handle_connect 0x7f806c15e2a0 2026-02-25T15:07:55.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.274+0000 7f8059ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:55.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3341298525 0 0) 0x7f806c199570 con 0x7f806c059e10 2026-02-25T15:07:55.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.285 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 10 monclient(hunting): my global_id is 27442 2026-02-25T15:07:55.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f803c003610 con 0x7f806c059e10 2026-02-25T15:07:55.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 670776118 0 0) 0x7f806c12a9e0 con 0x7f806c1a5660 2026-02-25T15:07:55.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 10 monclient(hunting): my global_id is 28580 2026-02-25T15:07:55.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f806c199570 con 0x7f806c1a5660 2026-02-25T15:07:55.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3824667432 0 0) 0x7f806c1630d0 con 0x7f806c15e2a0 2026-02-25T15:07:55.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.286 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 10 monclient(hunting): my global_id is 30810 2026-02-25T15:07:55.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.275+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f806c12a9e0 con 0x7f806c15e2a0 2026-02-25T15:07:55.289 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.277+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2892068079 0 0) 0x7f803c003610 con 0x7f806c059e10 2026-02-25T15:07:55.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.277+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f806c1630d0 con 0x7f806c059e10 2026-02-25T15:07:55.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.277+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f805c0030c0 con 0x7f806c059e10 2026-02-25T15:07:55.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.277+0000 7f8059ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:55.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.277+0000 7f8059ffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:55.290 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.277+0000 7f8059ffb640 10 monclient(hunting): dump: 2026-02-25T15:07:55.290 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.290 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.290 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.290 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.290 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.290 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.291 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.291 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.291 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.291 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.277+0000 7f8059ffb640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:55.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.277+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 32748354 0 0) 0x7f806c199570 con 0x7f806c1a5660 2026-02-25T15:07:55.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.277+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f803c003610 con 0x7f806c1a5660 2026-02-25T15:07:55.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.277+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8068003690 con 0x7f806c1a5660 2026-02-25T15:07:55.291 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.277+0000 7f8059ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:55.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.277+0000 7f8059ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:55.292 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.277+0000 7f8059ffb640 10 monclient(hunting): dump: 2026-02-25T15:07:55.292 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.292 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.292 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.292 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.292 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.292 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.293 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.293 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.293 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.293 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.278+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3640542379 0 0) 0x7f806c12a9e0 con 0x7f806c15e2a0 2026-02-25T15:07:55.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.278+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f806c199570 con 0x7f806c15e2a0 2026-02-25T15:07:55.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.278+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8060002ff0 con 0x7f806c15e2a0 2026-02-25T15:07:55.293 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.278+0000 7f8059ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:55.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.278+0000 7f8059ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:55.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.278+0000 7f8059ffb640 10 monclient(hunting): dump: 2026-02-25T15:07:55.294 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.294 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.294 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.294 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.294 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.294 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.295 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.295 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.295 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.295 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 227081311 0 0) 0x7f806c1630d0 con 0x7f806c059e10 2026-02-25T15:07:55.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 >> v1:10.20.193.94:6789/0 conn(0x7f806c1a5660 legacy=0x7f806c198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 >> v1:10.20.193.90:6789/0 conn(0x7f806c15e2a0 legacy=0x7f806c12a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8059ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:07:55.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8059ffb640 1 monclient: found mon.b 2026-02-25T15:07:55.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8059ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:55.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8059ffb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:55.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806c12acc0 con 0x7f806c059e10 2026-02-25T15:07:55.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8059ffb640 10 monclient: _finish_auth 0 2026-02-25T15:07:55.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8059ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:07:55.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8059ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:55.296 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8073855640 5 monclient: authenticate success, global_id 27442 2026-02-25T15:07:55.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8073855640 10 monclient: _renew_subs 2026-02-25T15:07:55.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8073855640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:55.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8073855640 1 -- 10.20.193.90:0/3957528640 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f806c05b8d0 con 0x7f806c059e10 2026-02-25T15:07:55.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8073855640 10 client.27442.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:55.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8073855640 10 monclient: _renew_subs 2026-02-25T15:07:55.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8073855640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:55.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8073855640 1 -- 10.20.193.90:0/3957528640 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f806c05be90 con 0x7f806c059e10 2026-02-25T15:07:55.297 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8073855640 1 librados: init done 2026-02-25T15:07:55.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8073855640 10 client.27442.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:55.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8073855640 15 client.27442.objecter _session_command_op_assign -1 1 2026-02-25T15:07:55.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8073855640 20 client.27442.objecter _get_session osd=-1 returning homeless 2026-02-25T15:07:55.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8073855640 10 client.27442.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:55.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8073855640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:55.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.279+0000 7f8073855640 1 -- 10.20.193.90:0/3957528640 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f806c12afa0 con 0x7f806c059e10 2026-02-25T15:07:55.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.287+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f805c003490 con 0x7f806c059e10 2026-02-25T15:07:55.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.287+0000 7f8059ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:55.298 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.287+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f805c0039b0 con 0x7f806c059e10 2026-02-25T15:07:55.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.287+0000 7f8059ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:55.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.287+0000 7f8059ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:55.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.287+0000 7f805b7fe640 4 set_mon_vals no callback set 2026-02-25T15:07:55.299 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.287+0000 7f8059ffb640 10 monclient: dump: 2026-02-25T15:07:55.299 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.299 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.299 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.299 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.300 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.300 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.300 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.300 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.300 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.300 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.300 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.288+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f805c003b90 con 0x7f806c059e10 2026-02-25T15:07:55.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8070dc9640 10 monclient: get_auth_request con 0x7f803c07a080 auth_method 0 2026-02-25T15:07:55.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(337..337 src has 1..337) ==== 8959+0+0 (unknown 3556021173 0 0) 0x7f805c095b10 con 0x7f806c059e10 2026-02-25T15:07:55.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 10 client.27442.objecter ms_dispatch2 0x7f806c000b90 osd_map(337..337 src has 1..337) 2026-02-25T15:07:55.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 3 client.27442.objecter handle_osd_map got epochs [337,337] > 0 2026-02-25T15:07:55.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 3 client.27442.objecter handle_osd_map decoding full epoch 337 2026-02-25T15:07:55.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 10 client.27442.objecter checking command 1 2026-02-25T15:07:55.303 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 20 client.27442.objecter _get_session s=0x7f803c0a4330 osd=8 3 2026-02-25T15:07:55.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 20 client.27442.objecter put_session s=0x7f803c0a4330 osd=8 3 2026-02-25T15:07:55.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 15 client.27442.objecter _session_command_op_remove -1 1 2026-02-25T15:07:55.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 20 client.27442.objecter _get_session s=0x7f803c0a4330 osd=8 3 2026-02-25T15:07:55.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 20 client.27442.objecter get_session s=0x7f803c0a4330 osd=8 3 2026-02-25T15:07:55.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 15 client.27442.objecter _session_command_op_assign 8 1 2026-02-25T15:07:55.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 20 client.27442.objecter put_session s=0x7f803c0a4330 osd=8 4 2026-02-25T15:07:55.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 10 client.27442.objecter _send_command 1 2026-02-25T15:07:55.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 --> v1:10.20.193.90:6801/2320036239 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f803c0c66e0 con 0x7f803c0a49e0 2026-02-25T15:07:55.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 20 client.27442.objecter dump_active .. 0 homeless 2026-02-25T15:07:55.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 10 client.27442.objecter ms_handle_connect 0x7f803c07a080 2026-02-25T15:07:55.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_get_version_reply(handle=1 version=337) ==== 24+0+0 (unknown 676316437 0 0) 0x7f805c05eb40 con 0x7f806c059e10 2026-02-25T15:07:55.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8059ffb640 10 monclient: handle_get_version_reply finishing 1 version 337 2026-02-25T15:07:55.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.289+0000 7f8071dcb640 10 monclient: get_auth_request con 0x7f803c0a49e0 auth_method 0 2026-02-25T15:07:55.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.290+0000 7f8059ffb640 10 client.27442.objecter ms_handle_connect 0x7f803c0a49e0 2026-02-25T15:07:55.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.299+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== osd.8 v1:10.20.193.90:6801/2320036239 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f803c0c66e0 con 0x7f803c0a49e0 2026-02-25T15:07:55.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.299+0000 7f8059ffb640 10 client.27442.objecter ms_dispatch2 0x7f806c000b90 command_reply(tid 1: 0 ) 2026-02-25T15:07:55.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.299+0000 7f8059ffb640 10 client.27442.objecter _finish_command 1 = system:0 2026-02-25T15:07:55.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.299+0000 7f8059ffb640 20 client.27442.objecter put_session s=0x7f803c0a4330 osd=8 4 2026-02-25T15:07:55.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.299+0000 7f8059ffb640 15 client.27442.objecter _session_command_op_remove 8 1 2026-02-25T15:07:55.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.306+0000 7f8073855640 10 client.27442.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:07:55.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.306+0000 7f8073855640 15 client.27442.objecter _session_command_op_assign -1 2 2026-02-25T15:07:55.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.306+0000 7f8073855640 20 client.27442.objecter _get_session s=0x7f803c0a4330 osd=8 3 2026-02-25T15:07:55.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.306+0000 7f8073855640 20 client.27442.objecter put_session s=0x7f803c0a4330 osd=8 3 2026-02-25T15:07:55.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.306+0000 7f8073855640 20 client.27442.objecter _get_session s=0x7f803c0a4330 osd=8 3 2026-02-25T15:07:55.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.306+0000 7f8073855640 15 client.27442.objecter _session_command_op_remove -1 2 2026-02-25T15:07:55.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.306+0000 7f8073855640 20 client.27442.objecter get_session s=0x7f803c0a4330 osd=8 3 2026-02-25T15:07:55.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.306+0000 7f8073855640 15 client.27442.objecter _session_command_op_assign 8 2 2026-02-25T15:07:55.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.306+0000 7f8073855640 20 client.27442.objecter put_session s=0x7f803c0a4330 osd=8 4 2026-02-25T15:07:55.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.306+0000 7f8073855640 10 client.27442.objecter _send_command 2 2026-02-25T15:07:55.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.306+0000 7f8073855640 1 -- 10.20.193.90:0/3957528640 --> v1:10.20.193.90:6801/2320036239 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f806c0531e0 con 0x7f803c0a49e0 2026-02-25T15:07:55.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8059ffb640 1 -- 10.20.193.90:0/3957528640 <== osd.8 v1:10.20.193.90:6801/2320036239 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f806c0531e0 con 0x7f803c0a49e0 2026-02-25T15:07:55.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8059ffb640 10 client.27442.objecter ms_dispatch2 0x7f806c000b90 command_reply(tid 2: 0 ) 2026-02-25T15:07:55.315 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8059ffb640 10 client.27442.objecter _finish_command 2 = system:0 2026-02-25T15:07:55.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8059ffb640 20 client.27442.objecter put_session s=0x7f803c0a4330 osd=8 4 2026-02-25T15:07:55.316 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8059ffb640 15 client.27442.objecter _session_command_op_remove 8 2 2026-02-25T15:07:55.316 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:07:55.316 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-25T15:07:55.316 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:07:55.317 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 10 librados: watch_flush enter 2026-02-25T15:07:55.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 10 librados: watch_flush exit 2026-02-25T15:07:55.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 10 client.27442.objecter close_session for osd.8 2026-02-25T15:07:55.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 1 -- 10.20.193.90:0/3957528640 >> v1:10.20.193.90:6801/2320036239 conn(0x7f803c0a49e0 legacy=0x7f803c0c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 20 client.27442.objecter put_session s=0x7f803c0a4330 osd=8 1 2026-02-25T15:07:55.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 20 client.27442.objecter shutdown clearing up homeless session... 2026-02-25T15:07:55.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 10 client.27442.objecter successfully canceled tick 2026-02-25T15:07:55.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 1 -- 10.20.193.90:0/3957528640 >> v1:10.20.193.189:6800/3647375982 conn(0x7f803c07a080 legacy=0x7f803c09a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 10 monclient: shutdown 2026-02-25T15:07:55.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:55.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 1 -- 10.20.193.90:0/3957528640 >> v1:10.20.193.93:6789/0 conn(0x7f806c059e10 legacy=0x7f806c05c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.318 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8071dcb640 1 -- 10.20.193.90:0/3957528640 reap_dead start 2026-02-25T15:07:55.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 1 -- 10.20.193.90:0/3957528640 shutdown_connections 2026-02-25T15:07:55.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 1 -- 10.20.193.90:0/3957528640 >> 10.20.193.90:0/3957528640 conn(0x7f806c0b2f60 msgr2=0x7f806c164960 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:55.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 1 -- 10.20.193.90:0/3957528640 shutdown_connections 2026-02-25T15:07:55.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 1 -- 10.20.193.90:0/3957528640 wait complete. 2026-02-25T15:07:55.319 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.314+0000 7f8073855640 1 librados: shutdown 2026-02-25T15:07:55.323 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2026-02-25T15:07:55.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.388+0000 7f55fd8e1640 10 monclient: get_monmap_and_config 2026-02-25T15:07:55.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.388+0000 7f55fd8e1640 10 monclient: build_initial_monmap 2026-02-25T15:07:55.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.388+0000 7f55fd8e1640 10 monclient: monmap: 2026-02-25T15:07:55.390 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:55.390 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.390 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:55.389186+0000 2026-02-25T15:07:55.390 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:55.389186+0000 2026-02-25T15:07:55.390 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:55.391 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.391 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:55.391 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:55.391 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:55.391 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.389+0000 7f55fd8e1640 1 Processor -- start 2026-02-25T15:07:55.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.389+0000 7f55fd8e1640 1 -- start start 2026-02-25T15:07:55.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.389+0000 7f55fd8e1640 10 monclient: init 2026-02-25T15:07:55.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.389+0000 7f55fd8e1640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:55.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.389+0000 7f55fd8e1640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:07:55.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.389+0000 7f55fd8e1640 10 monclient(hunting): picked mon.noname-c con 0x7f55f81566e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:55.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.389+0000 7f55fd8e1640 10 monclient(hunting): picked mon.noname-b con 0x7f55f8160c00 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:55.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.389+0000 7f55fd8e1640 10 monclient(hunting): picked mon.noname-a con 0x7f55f8156ab0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:55.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.389+0000 7f55fd8e1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f8058480 con 0x7f55f8156ab0 2026-02-25T15:07:55.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.389+0000 7f55fd8e1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f8058760 con 0x7f55f8160c00 2026-02-25T15:07:55.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.389+0000 7f55fd8e1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f80b0f90 con 0x7f55f81566e0 2026-02-25T15:07:55.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.389+0000 7f55fd8e1640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:55.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.389+0000 7f55fd8e1640 10 monclient(hunting): authenticate will time out at 3364.989596s 2026-02-25T15:07:55.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.389+0000 7f55f67fc640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f55f8160c00 0x7f55f8057920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45516/0 (socket says 10.20.193.90:45516) 2026-02-25T15:07:55.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.389+0000 7f55f67fc640 1 -- 10.20.193.90:0/3650019827 learned_addr learned my addr 10.20.193.90:0/3650019827 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:55.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 1 -- 10.20.193.90:0/3650019827 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2300730962 0 0) 0x7f55f8058760 con 0x7f55f8160c00 2026-02-25T15:07:55.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 10 monclient(hunting): my global_id is 27445 2026-02-25T15:07:55.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 1 -- 10.20.193.90:0/3650019827 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55e4003610 con 0x7f55f8160c00 2026-02-25T15:07:55.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 1 -- 10.20.193.90:0/3650019827 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3354247725 0 0) 0x7f55f80b0f90 con 0x7f55f81566e0 2026-02-25T15:07:55.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 10 monclient(hunting): my global_id is 28583 2026-02-25T15:07:55.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 1 -- 10.20.193.90:0/3650019827 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55f8058760 con 0x7f55f81566e0 2026-02-25T15:07:55.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 1 -- 10.20.193.90:0/3650019827 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2671779456 0 0) 0x7f55f8058480 con 0x7f55f8156ab0 2026-02-25T15:07:55.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 10 monclient(hunting): my global_id is 30813 2026-02-25T15:07:55.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.391+0000 7f55f5ffb640 1 -- 10.20.193.90:0/3650019827 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55f80b0f90 con 0x7f55f8156ab0 2026-02-25T15:07:55.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55f5ffb640 1 -- 10.20.193.90:0/3650019827 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2486416588 0 0) 0x7f55e4003610 con 0x7f55f8160c00 2026-02-25T15:07:55.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55f5ffb640 1 -- 10.20.193.90:0/3650019827 >> v1:10.20.193.94:6789/0 conn(0x7f55f81566e0 legacy=0x7f55f80b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55f5ffb640 1 -- 10.20.193.90:0/3650019827 >> v1:10.20.193.90:6789/0 conn(0x7f55f8156ab0 legacy=0x7f55f8055980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55f5ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:07:55.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55f5ffb640 1 monclient: found mon.noname-b 2026-02-25T15:07:55.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55f5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:55.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55f5ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:07:55.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55f5ffb640 1 -- 10.20.193.90:0/3650019827 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f80b1270 con 0x7f55f8160c00 2026-02-25T15:07:55.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55f5ffb640 10 monclient: _finish_auth 0 2026-02-25T15:07:55.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55f5ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:07:55.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55f5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:55.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55f5ffb640 1 -- 10.20.193.90:0/3650019827 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f55e0002df0 con 0x7f55f8160c00 2026-02-25T15:07:55.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:55.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55f5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:55.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55f5ffb640 10 monclient: dump: 2026-02-25T15:07:55.404 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.404 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.404 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.404 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.404 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.404 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.404 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.404 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.405 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.405 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55fd8e1640 5 monclient: authenticate success, global_id 27445 2026-02-25T15:07:55.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.392+0000 7f55fd8e1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:55.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.393+0000 7f55f5ffb640 1 -- 10.20.193.90:0/3650019827 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f55e0002fc0 con 0x7f55f8160c00 2026-02-25T15:07:55.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.393+0000 7f55f5ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:55.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.393+0000 7f55f5ffb640 1 -- 10.20.193.90:0/3650019827 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f55e0003400 con 0x7f55f8160c00 2026-02-25T15:07:55.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.393+0000 7f55f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:55.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.393+0000 7f55f5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:55.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.393+0000 7f55f5ffb640 10 monclient: dump: 2026-02-25T15:07:55.405 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.405 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.406 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.406 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.406 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.406 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.406 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.406 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.406 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.406 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.393+0000 7f55fd8e1640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:55.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.393+0000 7f55fd8e1640 4 set_mon_vals no callback set 2026-02-25T15:07:55.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.393+0000 7f55fd8e1640 10 monclient: shutdown 2026-02-25T15:07:55.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.393+0000 7f55fd8e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:55.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.393+0000 7f55fd8e1640 1 -- 10.20.193.90:0/3650019827 >> v1:10.20.193.93:6789/0 conn(0x7f55f8160c00 legacy=0x7f55f8057920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.393+0000 7f55fd8e1640 1 -- 10.20.193.90:0/3650019827 shutdown_connections 2026-02-25T15:07:55.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.393+0000 7f55fd8e1640 1 -- 10.20.193.90:0/3650019827 >> 10.20.193.90:0/3650019827 conn(0x7f55f80b2ef0 msgr2=0x7f55f80b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:55.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.393+0000 7f55fd8e1640 1 -- 10.20.193.90:0/3650019827 shutdown_connections 2026-02-25T15:07:55.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.393+0000 7f55fd8e1640 1 -- 10.20.193.90:0/3650019827 wait complete. 2026-02-25T15:07:55.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.394+0000 7f55fd8e1640 10 monclient: build_initial_monmap 2026-02-25T15:07:55.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.394+0000 7f55fd8e1640 10 monclient: monmap: 2026-02-25T15:07:55.407 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:55.407 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:55.407 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:55.407 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:55.407 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:55.408 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:55.408 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:55.408 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:55.408 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:55.408 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.394+0000 7f55fd8e1640 1 librados: starting msgr at 2026-02-25T15:07:55.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.394+0000 7f55fd8e1640 1 librados: starting objecter 2026-02-25T15:07:55.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.394+0000 7f55fd8e1640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:55.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.394+0000 7f55fd8e1640 1 Processor -- start 2026-02-25T15:07:55.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.394+0000 7f55fd8e1640 1 -- start start 2026-02-25T15:07:55.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.394+0000 7f55fd8e1640 1 librados: setting wanted keys 2026-02-25T15:07:55.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.394+0000 7f55fd8e1640 1 librados: calling monclient init 2026-02-25T15:07:55.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.394+0000 7f55fd8e1640 10 monclient: init 2026-02-25T15:07:55.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55fd8e1640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:55.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55fd8e1640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:07:55.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55fd8e1640 10 monclient(hunting): picked mon.noname-a con 0x7f55f81566e0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:55.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55fd8e1640 10 monclient(hunting): picked mon.noname-b con 0x7f55f8156ab0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:55.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55fd8e1640 10 monclient(hunting): picked mon.noname-c con 0x7f55f8160c00 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:55.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55fd8e1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f8158ba0 con 0x7f55f81566e0 2026-02-25T15:07:55.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55fd8e1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f8158e80 con 0x7f55f8156ab0 2026-02-25T15:07:55.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55fd8e1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f8159160 con 0x7f55f8160c00 2026-02-25T15:07:55.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55fd8e1640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:55.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55fd8e1640 10 monclient(hunting): authenticate will time out at 3364.995264s 2026-02-25T15:07:55.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55f67fc640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f55f8156ab0 0x7f55f812b2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45520/0 (socket says 10.20.193.90:45520) 2026-02-25T15:07:55.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55f67fc640 1 -- 10.20.193.90:0/663377648 learned_addr learned my addr 10.20.193.90:0/663377648 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:55.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55cffff640 10 client.?.objecter ms_handle_connect 0x7f55f8156ab0 2026-02-25T15:07:55.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55cffff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:55.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55cffff640 10 client.?.objecter ms_handle_connect 0x7f55f81566e0 2026-02-25T15:07:55.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55cffff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:55.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55cffff640 10 client.?.objecter ms_handle_connect 0x7f55f8160c00 2026-02-25T15:07:55.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.395+0000 7f55cffff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:55.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3552922298 0 0) 0x7f55f8158e80 con 0x7f55f8156ab0 2026-02-25T15:07:55.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 10 monclient(hunting): my global_id is 27448 2026-02-25T15:07:55.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55c8003610 con 0x7f55f8156ab0 2026-02-25T15:07:55.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3061764707 0 0) 0x7f55f8158ba0 con 0x7f55f81566e0 2026-02-25T15:07:55.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 10 monclient(hunting): my global_id is 30816 2026-02-25T15:07:55.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55f8158e80 con 0x7f55f81566e0 2026-02-25T15:07:55.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3692598334 0 0) 0x7f55f8159160 con 0x7f55f8160c00 2026-02-25T15:07:55.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 10 monclient(hunting): my global_id is 28586 2026-02-25T15:07:55.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.396+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55f8158ba0 con 0x7f55f8160c00 2026-02-25T15:07:55.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.398+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 821230171 0 0) 0x7f55c8003610 con 0x7f55f8156ab0 2026-02-25T15:07:55.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.398+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f55f8159160 con 0x7f55f8156ab0 2026-02-25T15:07:55.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.398+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f55e0003310 con 0x7f55f8156ab0 2026-02-25T15:07:55.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.398+0000 7f55cffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:55.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.398+0000 7f55cffff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:55.412 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.398+0000 7f55cffff640 10 monclient(hunting): dump: 2026-02-25T15:07:55.412 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.412 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.412 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.412 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.412 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.412 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.412 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.413 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.413 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.413 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.398+0000 7f55cffff640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:55.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.399+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1601026767 0 0) 0x7f55f8158ba0 con 0x7f55f8160c00 2026-02-25T15:07:55.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.399+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f55c8003610 con 0x7f55f8160c00 2026-02-25T15:07:55.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.399+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f55e8003450 con 0x7f55f8160c00 2026-02-25T15:07:55.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.399+0000 7f55cffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:55.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.399+0000 7f55cffff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:55.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.399+0000 7f55cffff640 10 monclient(hunting): dump: 2026-02-25T15:07:55.413 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.413 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.413 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.414 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.414 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.414 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.414 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.414 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.414 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.414 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4287596102 0 0) 0x7f55f8158e80 con 0x7f55f81566e0 2026-02-25T15:07:55.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f55f8158ba0 con 0x7f55f81566e0 2026-02-25T15:07:55.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f55ec0030b0 con 0x7f55f81566e0 2026-02-25T15:07:55.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:55.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:55.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 10 monclient(hunting): dump: 2026-02-25T15:07:55.415 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.415 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.415 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.415 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.415 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.415 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.415 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.415 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.415 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.416 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 356953838 0 0) 0x7f55f8159160 con 0x7f55f8156ab0 2026-02-25T15:07:55.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 >> v1:10.20.193.94:6789/0 conn(0x7f55f8160c00 legacy=0x7f55f812c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 >> v1:10.20.193.90:6789/0 conn(0x7f55f81566e0 legacy=0x7f55f80a96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 10 monclient: _finish_hunting 0 2026-02-25T15:07:55.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 1 monclient: found mon.b 2026-02-25T15:07:55.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:55.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:55.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f812a7d0 con 0x7f55f8156ab0 2026-02-25T15:07:55.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 10 monclient: _finish_auth 0 2026-02-25T15:07:55.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 10 monclient: _check_auth_tickets 2026-02-25T15:07:55.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55cffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:55.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55fd8e1640 5 monclient: authenticate success, global_id 27448 2026-02-25T15:07:55.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55fd8e1640 10 monclient: _renew_subs 2026-02-25T15:07:55.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55fd8e1640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:55.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55fd8e1640 1 -- 10.20.193.90:0/663377648 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f55f812aab0 con 0x7f55f8156ab0 2026-02-25T15:07:55.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55fd8e1640 10 client.27448.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:55.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55fd8e1640 10 monclient: _renew_subs 2026-02-25T15:07:55.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55fd8e1640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:55.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55fd8e1640 1 -- 10.20.193.90:0/663377648 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f55f812b9c0 con 0x7f55f8156ab0 2026-02-25T15:07:55.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.400+0000 7f55fd8e1640 1 librados: init done 2026-02-25T15:07:55.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.401+0000 7f55fd8e1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:55.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.401+0000 7f55fd8e1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:55.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.401+0000 7f55fd8e1640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:55.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.401+0000 7f55fd8e1640 1 -- 10.20.193.90:0/663377648 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f815a520 con 0x7f55f8156ab0 2026-02-25T15:07:55.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.408+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f55e0002960 con 0x7f55f8156ab0 2026-02-25T15:07:55.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.408+0000 7f55cffff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:55.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.408+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f55e0004b80 con 0x7f55f8156ab0 2026-02-25T15:07:55.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.408+0000 7f55cffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:55.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.408+0000 7f55cffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:55.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.408+0000 7f55cffff640 10 monclient: dump: 2026-02-25T15:07:55.420 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.420 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.420 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.420 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.420 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.420 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.420 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.420 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.421 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.421 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.408+0000 7f55f57fa640 4 set_mon_vals no callback set 2026-02-25T15:07:55.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.409+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f55e001e060 con 0x7f55f8156ab0 2026-02-25T15:07:55.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.409+0000 7f55f6ffd640 10 monclient: get_auth_request con 0x7f55c8079fa0 auth_method 0 2026-02-25T15:07:55.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.410+0000 7f55cffff640 10 client.27448.objecter ms_handle_connect 0x7f55c8079fa0 2026-02-25T15:07:55.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.410+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(337..337 src has 1..337) ==== 8959+0+0 (unknown 3556021173 0 0) 0x7f55e0095990 con 0x7f55f8156ab0 2026-02-25T15:07:55.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.410+0000 7f55cffff640 10 client.27448.objecter ms_dispatch2 0x7f55f8000b90 osd_map(337..337 src has 1..337) 2026-02-25T15:07:55.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.410+0000 7f55cffff640 3 client.27448.objecter handle_osd_map got epochs [337,337] > 0 2026-02-25T15:07:55.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.410+0000 7f55cffff640 3 client.27448.objecter handle_osd_map decoding full epoch 337 2026-02-25T15:07:55.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.410+0000 7f55cffff640 20 client.27448.objecter dump_active .. 0 homeless 2026-02-25T15:07:55.495 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.494+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f55e005f2f0 con 0x7f55f8156ab0 2026-02-25T15:07:55.496 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.494+0000 7f55cffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:55.496 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.494+0000 7f55cffff640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:55.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.599+0000 7f55fd8e1640 1 -- 10.20.193.90:0/663377648 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "8", "target": ["mon-mgr", ""]}) -- 0x7f55f805e280 con 0x7f55c8079fa0 2026-02-25T15:07:55.601 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.600+0000 7f55cffff640 1 -- 10.20.193.90:0/663377648 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 8 to deep-scrub) ==== 41+0+0 (unknown 1222854447 0 0) 0x7f55f805e280 con 0x7f55c8079fa0 2026-02-25T15:07:55.602 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 8 to deep-scrub 2026-02-25T15:07:55.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.601+0000 7f55fd8e1640 10 librados: watch_flush enter 2026-02-25T15:07:55.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.601+0000 7f55fd8e1640 10 librados: watch_flush exit 2026-02-25T15:07:55.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.601+0000 7f55fd8e1640 20 client.27448.objecter shutdown clearing up homeless session... 2026-02-25T15:07:55.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.601+0000 7f55fd8e1640 10 client.27448.objecter successfully canceled tick 2026-02-25T15:07:55.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.601+0000 7f55fd8e1640 1 -- 10.20.193.90:0/663377648 >> v1:10.20.193.189:6800/3647375982 conn(0x7f55c8079fa0 legacy=0x7f55c809a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.601+0000 7f55fd8e1640 10 monclient: shutdown 2026-02-25T15:07:55.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.601+0000 7f55fd8e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:55.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.601+0000 7f55fd8e1640 1 -- 10.20.193.90:0/663377648 >> v1:10.20.193.93:6789/0 conn(0x7f55f8156ab0 legacy=0x7f55f812b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.601+0000 7f55fd8e1640 1 -- 10.20.193.90:0/663377648 shutdown_connections 2026-02-25T15:07:55.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.601+0000 7f55fd8e1640 1 -- 10.20.193.90:0/663377648 >> 10.20.193.90:0/663377648 conn(0x7f55f80b2ef0 msgr2=0x7f55f8057850 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:55.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.601+0000 7f55fd8e1640 1 -- 10.20.193.90:0/663377648 shutdown_connections 2026-02-25T15:07:55.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.601+0000 7f55fd8e1640 1 -- 10.20.193.90:0/663377648 wait complete. 2026-02-25T15:07:55.604 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.601+0000 7f55fd8e1640 1 librados: shutdown 2026-02-25T15:07:55.613 INFO:tasks.ceph:Scrubbing osd.12 2026-02-25T15:07:55.613 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:07:55.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.682+0000 7fa5defc0640 10 monclient: get_monmap_and_config 2026-02-25T15:07:55.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.682+0000 7fa5defc0640 10 monclient: build_initial_monmap 2026-02-25T15:07:55.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.682+0000 7fa5defc0640 10 monclient: monmap: 2026-02-25T15:07:55.685 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:55.685 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.685 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:55.683561+0000 2026-02-25T15:07:55.685 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:55.683561+0000 2026-02-25T15:07:55.685 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:55.685 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.686 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:55.686 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:55.686 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:55.686 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.686 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.683+0000 7fa5defc0640 1 Processor -- start 2026-02-25T15:07:55.686 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.683+0000 7fa5defc0640 1 -- start start 2026-02-25T15:07:55.686 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.683+0000 7fa5defc0640 10 monclient: init 2026-02-25T15:07:55.693 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.683+0000 7fa5defc0640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:55.693 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.683+0000 7fa5defc0640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:07:55.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.683+0000 7fa5defc0640 10 monclient(hunting): picked mon.noname-b con 0x7fa5d815e2a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:55.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.683+0000 7fa5defc0640 10 monclient(hunting): picked mon.noname-a con 0x7fa5d8059e10 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:55.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.683+0000 7fa5defc0640 10 monclient(hunting): picked mon.noname-c con 0x7fa5d81a5660 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:55.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.683+0000 7fa5defc0640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5d8162df0 con 0x7fa5d8059e10 2026-02-25T15:07:55.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.683+0000 7fa5defc0640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5d81630d0 con 0x7fa5d815e2a0 2026-02-25T15:07:55.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.683+0000 7fa5defc0640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5d81633b0 con 0x7fa5d81a5660 2026-02-25T15:07:55.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.683+0000 7fa5defc0640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:55.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.683+0000 7fa5defc0640 10 monclient(hunting): authenticate will time out at 3365.283911s 2026-02-25T15:07:55.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.684+0000 7fa5dcd35640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fa5d815e2a0 0x7fa5d805da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45528/0 (socket says 10.20.193.90:45528) 2026-02-25T15:07:55.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.684+0000 7fa5dcd35640 1 -- 10.20.193.90:0/1294582090 learned_addr learned my addr 10.20.193.90:0/1294582090 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:55.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.685+0000 7fa5cf7fe640 1 -- 10.20.193.90:0/1294582090 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 651565234 0 0) 0x7fa5d81633b0 con 0x7fa5d81a5660 2026-02-25T15:07:55.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.685+0000 7fa5cf7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.685+0000 7fa5cf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.685+0000 7fa5cf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.685+0000 7fa5cf7fe640 10 monclient(hunting): my global_id is 28589 2026-02-25T15:07:55.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.685+0000 7fa5cf7fe640 1 -- 10.20.193.90:0/1294582090 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5c4003610 con 0x7fa5d81a5660 2026-02-25T15:07:55.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.685+0000 7fa5cf7fe640 1 -- 10.20.193.90:0/1294582090 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268933474 0 0) 0x7fa5d81630d0 con 0x7fa5d815e2a0 2026-02-25T15:07:55.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.685+0000 7fa5cf7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.685+0000 7fa5cf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.685+0000 7fa5cf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.685+0000 7fa5cf7fe640 10 monclient(hunting): my global_id is 27451 2026-02-25T15:07:55.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.685+0000 7fa5cf7fe640 1 -- 10.20.193.90:0/1294582090 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5d81633b0 con 0x7fa5d815e2a0 2026-02-25T15:07:55.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.686+0000 7fa5cf7fe640 1 -- 10.20.193.90:0/1294582090 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1118069790 0 0) 0x7fa5d8162df0 con 0x7fa5d8059e10 2026-02-25T15:07:55.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.686+0000 7fa5cf7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.686+0000 7fa5cf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.686+0000 7fa5cf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.686+0000 7fa5cf7fe640 10 monclient(hunting): my global_id is 30819 2026-02-25T15:07:55.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.686+0000 7fa5cf7fe640 1 -- 10.20.193.90:0/1294582090 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5d81630d0 con 0x7fa5d8059e10 2026-02-25T15:07:55.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5cf7fe640 1 -- 10.20.193.90:0/1294582090 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2609666996 0 0) 0x7fa5c4003610 con 0x7fa5d81a5660 2026-02-25T15:07:55.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5cf7fe640 1 -- 10.20.193.90:0/1294582090 >> v1:10.20.193.93:6789/0 conn(0x7fa5d815e2a0 legacy=0x7fa5d805da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5cf7fe640 1 -- 10.20.193.90:0/1294582090 >> v1:10.20.193.90:6789/0 conn(0x7fa5d8059e10 legacy=0x7fa5d815ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5cf7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:07:55.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5cf7fe640 1 monclient: found mon.noname-c 2026-02-25T15:07:55.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:55.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5cf7fe640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:07:55.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5cf7fe640 1 -- 10.20.193.90:0/1294582090 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d8163690 con 0x7fa5d81a5660 2026-02-25T15:07:55.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5cf7fe640 10 monclient: _finish_auth 0 2026-02-25T15:07:55.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5cf7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:07:55.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:55.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5cf7fe640 1 -- 10.20.193.90:0/1294582090 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa5d4003170 con 0x7fa5d81a5660 2026-02-25T15:07:55.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:55.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5cf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:07:55.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5cf7fe640 10 monclient: dump: 2026-02-25T15:07:55.698 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.698 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.698 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.698 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.698 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.698 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.698 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.698 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.698 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.698 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5defc0640 5 monclient: authenticate success, global_id 28589 2026-02-25T15:07:55.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.687+0000 7fa5defc0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:55.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.688+0000 7fa5cf7fe640 1 -- 10.20.193.90:0/1294582090 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa5d4003340 con 0x7fa5d81a5660 2026-02-25T15:07:55.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.688+0000 7fa5cf7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:55.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.688+0000 7fa5cf7fe640 1 -- 10.20.193.90:0/1294582090 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa5d4003780 con 0x7fa5d81a5660 2026-02-25T15:07:55.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.688+0000 7fa5cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:55.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.688+0000 7fa5cf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:55.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.688+0000 7fa5cf7fe640 10 monclient: dump: 2026-02-25T15:07:55.699 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.699 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.700 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.700 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.700 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.700 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.700 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.700 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.700 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.700 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.700 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.688+0000 7fa5defc0640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:55.700 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.688+0000 7fa5defc0640 4 set_mon_vals no callback set 2026-02-25T15:07:55.700 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.688+0000 7fa5defc0640 10 monclient: shutdown 2026-02-25T15:07:55.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.688+0000 7fa5defc0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:55.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.688+0000 7fa5defc0640 1 -- 10.20.193.90:0/1294582090 >> v1:10.20.193.94:6789/0 conn(0x7fa5d81a5660 legacy=0x7fa5d81c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.688+0000 7fa5defc0640 1 -- 10.20.193.90:0/1294582090 shutdown_connections 2026-02-25T15:07:55.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.688+0000 7fa5defc0640 1 -- 10.20.193.90:0/1294582090 >> 10.20.193.90:0/1294582090 conn(0x7fa5d80b2f60 msgr2=0x7fa5d80b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:55.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.688+0000 7fa5defc0640 1 -- 10.20.193.90:0/1294582090 shutdown_connections 2026-02-25T15:07:55.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.688+0000 7fa5defc0640 1 -- 10.20.193.90:0/1294582090 wait complete. 2026-02-25T15:07:55.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 10 monclient: build_initial_monmap 2026-02-25T15:07:55.701 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 10 monclient: monmap: 2026-02-25T15:07:55.701 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:55.701 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:55.701 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:55.702 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:55.702 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:55.702 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:55.702 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:55.702 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:55.702 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:55.702 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.702 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 1 librados: starting msgr at 2026-02-25T15:07:55.702 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 1 librados: starting objecter 2026-02-25T15:07:55.702 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:55.702 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 1 Processor -- start 2026-02-25T15:07:55.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 1 -- start start 2026-02-25T15:07:55.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 1 librados: setting wanted keys 2026-02-25T15:07:55.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 1 librados: calling monclient init 2026-02-25T15:07:55.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 10 monclient: init 2026-02-25T15:07:55.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:55.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:07:55.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 10 monclient(hunting): picked mon.noname-b con 0x7fa5d8059e10 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:55.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 10 monclient(hunting): picked mon.noname-c con 0x7fa5d815e2a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:55.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 10 monclient(hunting): picked mon.noname-a con 0x7fa5d81a5660 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:55.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5d80b0bb0 con 0x7fa5d81a5660 2026-02-25T15:07:55.703 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5d8191b70 con 0x7fa5d8059e10 2026-02-25T15:07:55.704 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5d812af60 con 0x7fa5d815e2a0 2026-02-25T15:07:55.704 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:55.704 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.689+0000 7fa5defc0640 10 monclient(hunting): authenticate will time out at 3365.289898s 2026-02-25T15:07:55.704 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.690+0000 7fa5dcd35640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fa5d8059e10 0x7fa5d80a8d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45530/0 (socket says 10.20.193.90:45530) 2026-02-25T15:07:55.704 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.690+0000 7fa5dcd35640 1 -- 10.20.193.90:0/1430903331 learned_addr learned my addr 10.20.193.90:0/1430903331 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:55.704 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.690+0000 7fa5cd7fa640 10 client.?.objecter ms_handle_connect 0x7fa5d8059e10 2026-02-25T15:07:55.704 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.690+0000 7fa5cd7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:55.704 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.690+0000 7fa5cd7fa640 10 client.?.objecter ms_handle_connect 0x7fa5d815e2a0 2026-02-25T15:07:55.704 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.690+0000 7fa5cd7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:55.704 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.690+0000 7fa5cd7fa640 10 client.?.objecter ms_handle_connect 0x7fa5d81a5660 2026-02-25T15:07:55.704 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.690+0000 7fa5cd7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:55.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3378593569 0 0) 0x7fa5d812af60 con 0x7fa5d815e2a0 2026-02-25T15:07:55.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 10 monclient(hunting): my global_id is 28592 2026-02-25T15:07:55.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5a8003610 con 0x7fa5d815e2a0 2026-02-25T15:07:55.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3862173262 0 0) 0x7fa5d8191b70 con 0x7fa5d8059e10 2026-02-25T15:07:55.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.705 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 10 monclient(hunting): my global_id is 27454 2026-02-25T15:07:55.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5d812af60 con 0x7fa5d8059e10 2026-02-25T15:07:55.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 98577801 0 0) 0x7fa5d80b0bb0 con 0x7fa5d81a5660 2026-02-25T15:07:55.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 10 monclient(hunting): my global_id is 30822 2026-02-25T15:07:55.706 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.691+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5d8191b70 con 0x7fa5d81a5660 2026-02-25T15:07:55.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.693+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2949466361 0 0) 0x7fa5a8003610 con 0x7fa5d815e2a0 2026-02-25T15:07:55.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.693+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa5d80b0bb0 con 0x7fa5d815e2a0 2026-02-25T15:07:55.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.693+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa5c0002fb0 con 0x7fa5d815e2a0 2026-02-25T15:07:55.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.693+0000 7fa5cd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:55.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.693+0000 7fa5cd7fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:07:55.711 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.693+0000 7fa5cd7fa640 10 monclient(hunting): dump: 2026-02-25T15:07:55.711 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.711 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.712 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.712 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.712 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.712 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.712 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.712 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.712 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.712 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.693+0000 7fa5cd7fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:55.712 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.693+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1685610618 0 0) 0x7fa5d812af60 con 0x7fa5d8059e10 2026-02-25T15:07:55.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.693+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa5a8003610 con 0x7fa5d8059e10 2026-02-25T15:07:55.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.693+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa5c8003130 con 0x7fa5d8059e10 2026-02-25T15:07:55.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.693+0000 7fa5cd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:55.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.693+0000 7fa5cd7fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:55.713 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.693+0000 7fa5cd7fa640 10 monclient(hunting): dump: 2026-02-25T15:07:55.713 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.713 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.713 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.713 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.713 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.714 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.714 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.714 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.714 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.714 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.714 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.694+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2950038475 0 0) 0x7fa5d8191b70 con 0x7fa5d81a5660 2026-02-25T15:07:55.714 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.694+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa5d812af60 con 0x7fa5d81a5660 2026-02-25T15:07:55.714 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.694+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa5d40034f0 con 0x7fa5d81a5660 2026-02-25T15:07:55.714 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.694+0000 7fa5cd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:55.714 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.694+0000 7fa5cd7fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:55.714 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.694+0000 7fa5cd7fa640 10 monclient(hunting): dump: 2026-02-25T15:07:55.715 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.715 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.715 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.715 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.715 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.715 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.715 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.715 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.715 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.715 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.715 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1596098417 0 0) 0x7fa5d80b0bb0 con 0x7fa5d815e2a0 2026-02-25T15:07:55.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 >> v1:10.20.193.93:6789/0 conn(0x7fa5d8059e10 legacy=0x7fa5d80a8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 >> v1:10.20.193.90:6789/0 conn(0x7fa5d81a5660 legacy=0x7fa5d812a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5cd7fa640 10 monclient: _finish_hunting 0 2026-02-25T15:07:55.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5cd7fa640 1 monclient: found mon.c 2026-02-25T15:07:55.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5cd7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:55.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5cd7fa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:55.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d80b0e20 con 0x7fa5d815e2a0 2026-02-25T15:07:55.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5cd7fa640 10 monclient: _finish_auth 0 2026-02-25T15:07:55.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5cd7fa640 10 monclient: _check_auth_tickets 2026-02-25T15:07:55.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5cd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:55.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5defc0640 5 monclient: authenticate success, global_id 28592 2026-02-25T15:07:55.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5defc0640 10 monclient: _renew_subs 2026-02-25T15:07:55.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5defc0640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:55.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5defc0640 1 -- 10.20.193.90:0/1430903331 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d80b1100 con 0x7fa5d815e2a0 2026-02-25T15:07:55.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5defc0640 10 client.28592.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:55.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5defc0640 10 monclient: _renew_subs 2026-02-25T15:07:55.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5defc0640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:55.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5defc0640 1 -- 10.20.193.90:0/1430903331 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa5d812aaf0 con 0x7fa5d815e2a0 2026-02-25T15:07:55.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5defc0640 1 librados: init done 2026-02-25T15:07:55.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5defc0640 10 client.28592.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:55.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5defc0640 15 client.28592.objecter _session_command_op_assign -1 1 2026-02-25T15:07:55.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5defc0640 20 client.28592.objecter _get_session osd=-1 returning homeless 2026-02-25T15:07:55.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5defc0640 10 client.28592.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:55.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5defc0640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:55.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.695+0000 7fa5defc0640 1 -- 10.20.193.90:0/1430903331 --> v1:10.20.193.94:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fa5d8162df0 con 0x7fa5d815e2a0 2026-02-25T15:07:55.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.704+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa5d8162df0 con 0x7fa5d815e2a0 2026-02-25T15:07:55.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.704+0000 7fa5cd7fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:55.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.704+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa5c0003700 con 0x7fa5d815e2a0 2026-02-25T15:07:55.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.704+0000 7fa5cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:55.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.704+0000 7fa5cd7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:55.718 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.704+0000 7fa5cd7fa640 10 monclient: dump: 2026-02-25T15:07:55.718 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.718 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.718 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.718 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.718 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.718 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.719 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.719 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.719 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.719 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.719 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.704+0000 7fa5ceffd640 4 set_mon_vals no callback set 2026-02-25T15:07:55.719 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.705+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fa5c001cbe0 con 0x7fa5d815e2a0 2026-02-25T15:07:55.719 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.705+0000 7fa5dcd35640 10 monclient: get_auth_request con 0x7fa5a807a090 auth_method 0 2026-02-25T15:07:55.719 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.705+0000 7fa5cd7fa640 10 client.28592.objecter ms_handle_connect 0x7fa5a807a090 2026-02-25T15:07:55.719 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(337..337 src has 1..337) ==== 8959+0+0 (unknown 3556021173 0 0) 0x7fa5c0094780 con 0x7fa5d815e2a0 2026-02-25T15:07:55.719 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 10 client.28592.objecter ms_dispatch2 0x7fa5d8000b90 osd_map(337..337 src has 1..337) 2026-02-25T15:07:55.719 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 3 client.28592.objecter handle_osd_map got epochs [337,337] > 0 2026-02-25T15:07:55.719 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 3 client.28592.objecter handle_osd_map decoding full epoch 337 2026-02-25T15:07:55.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 10 client.28592.objecter checking command 1 2026-02-25T15:07:55.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 20 client.28592.objecter _get_session s=0x7fa5a80a4340 osd=12 3 2026-02-25T15:07:55.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 20 client.28592.objecter put_session s=0x7fa5a80a4340 osd=12 3 2026-02-25T15:07:55.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 15 client.28592.objecter _session_command_op_remove -1 1 2026-02-25T15:07:55.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 20 client.28592.objecter _get_session s=0x7fa5a80a4340 osd=12 3 2026-02-25T15:07:55.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 20 client.28592.objecter get_session s=0x7fa5a80a4340 osd=12 3 2026-02-25T15:07:55.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 15 client.28592.objecter _session_command_op_assign 12 1 2026-02-25T15:07:55.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 20 client.28592.objecter put_session s=0x7fa5a80a4340 osd=12 4 2026-02-25T15:07:55.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 10 client.28592.objecter _send_command 1 2026-02-25T15:07:55.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 --> v1:10.20.193.90:6812/1139796254 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5d4002bf0 con 0x7fa5a80a49f0 2026-02-25T15:07:55.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 20 client.28592.objecter dump_active .. 0 homeless 2026-02-25T15:07:55.720 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_get_version_reply(handle=1 version=337) ==== 24+0+0 (unknown 676316437 0 0) 0x7fa5c0059d50 con 0x7fa5d815e2a0 2026-02-25T15:07:55.721 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.706+0000 7fa5cd7fa640 10 monclient: handle_get_version_reply finishing 1 version 337 2026-02-25T15:07:55.721 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.707+0000 7fa5dd536640 10 monclient: get_auth_request con 0x7fa5a80a49f0 auth_method 0 2026-02-25T15:07:55.721 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.709+0000 7fa5cd7fa640 10 client.28592.objecter ms_handle_connect 0x7fa5a80a49f0 2026-02-25T15:07:55.722 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.720+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== osd.12 v1:10.20.193.90:6812/1139796254 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7fa5d4002bf0 con 0x7fa5a80a49f0 2026-02-25T15:07:55.722 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.720+0000 7fa5cd7fa640 10 client.28592.objecter ms_dispatch2 0x7fa5d8000b90 command_reply(tid 1: 0 ) 2026-02-25T15:07:55.722 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.720+0000 7fa5cd7fa640 10 client.28592.objecter _finish_command 1 = system:0 2026-02-25T15:07:55.722 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.720+0000 7fa5cd7fa640 20 client.28592.objecter put_session s=0x7fa5a80a4340 osd=12 4 2026-02-25T15:07:55.722 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.720+0000 7fa5cd7fa640 15 client.28592.objecter _session_command_op_remove 12 1 2026-02-25T15:07:55.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.727+0000 7fa5defc0640 10 client.28592.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:07:55.729 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.727+0000 7fa5defc0640 15 client.28592.objecter _session_command_op_assign -1 2 2026-02-25T15:07:55.729 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.727+0000 7fa5defc0640 20 client.28592.objecter _get_session s=0x7fa5a80a4340 osd=12 3 2026-02-25T15:07:55.729 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.727+0000 7fa5defc0640 20 client.28592.objecter put_session s=0x7fa5a80a4340 osd=12 3 2026-02-25T15:07:55.729 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.727+0000 7fa5defc0640 20 client.28592.objecter _get_session s=0x7fa5a80a4340 osd=12 3 2026-02-25T15:07:55.729 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.727+0000 7fa5defc0640 15 client.28592.objecter _session_command_op_remove -1 2 2026-02-25T15:07:55.729 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.727+0000 7fa5defc0640 20 client.28592.objecter get_session s=0x7fa5a80a4340 osd=12 3 2026-02-25T15:07:55.729 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.727+0000 7fa5defc0640 15 client.28592.objecter _session_command_op_assign 12 2 2026-02-25T15:07:55.729 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.727+0000 7fa5defc0640 20 client.28592.objecter put_session s=0x7fa5a80a4340 osd=12 4 2026-02-25T15:07:55.729 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.727+0000 7fa5defc0640 10 client.28592.objecter _send_command 2 2026-02-25T15:07:55.729 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.727+0000 7fa5defc0640 1 -- 10.20.193.90:0/1430903331 --> v1:10.20.193.90:6812/1139796254 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa5d80b2a20 con 0x7fa5a80a49f0 2026-02-25T15:07:55.736 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5cd7fa640 1 -- 10.20.193.90:0/1430903331 <== osd.12 v1:10.20.193.90:6812/1139796254 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7fa5d80b2a20 con 0x7fa5a80a49f0 2026-02-25T15:07:55.736 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5cd7fa640 10 client.28592.objecter ms_dispatch2 0x7fa5d8000b90 command_reply(tid 2: 0 ) 2026-02-25T15:07:55.737 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5cd7fa640 10 client.28592.objecter _finish_command 2 = system:0 2026-02-25T15:07:55.737 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5cd7fa640 20 client.28592.objecter put_session s=0x7fa5a80a4340 osd=12 4 2026-02-25T15:07:55.737 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5cd7fa640 15 client.28592.objecter _session_command_op_remove 12 2 2026-02-25T15:07:55.737 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:07:55.737 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-25T15:07:55.737 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:07:55.739 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 10 librados: watch_flush enter 2026-02-25T15:07:55.739 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 10 librados: watch_flush exit 2026-02-25T15:07:55.739 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 10 client.28592.objecter close_session for osd.12 2026-02-25T15:07:55.739 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 1 -- 10.20.193.90:0/1430903331 >> v1:10.20.193.90:6812/1139796254 conn(0x7fa5a80a49f0 legacy=0x7fa5a80c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.739 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 20 client.28592.objecter put_session s=0x7fa5a80a4340 osd=12 1 2026-02-25T15:07:55.739 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 20 client.28592.objecter shutdown clearing up homeless session... 2026-02-25T15:07:55.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 10 client.28592.objecter successfully canceled tick 2026-02-25T15:07:55.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 1 -- 10.20.193.90:0/1430903331 >> v1:10.20.193.189:6800/3647375982 conn(0x7fa5a807a090 legacy=0x7fa5a809a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 10 monclient: shutdown 2026-02-25T15:07:55.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:55.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 1 -- 10.20.193.90:0/1430903331 >> v1:10.20.193.94:6789/0 conn(0x7fa5d815e2a0 legacy=0x7fa5d8191460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5dd536640 1 -- 10.20.193.90:0/1430903331 reap_dead start 2026-02-25T15:07:55.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 1 -- 10.20.193.90:0/1430903331 shutdown_connections 2026-02-25T15:07:55.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 1 -- 10.20.193.90:0/1430903331 >> 10.20.193.90:0/1430903331 conn(0x7fa5d80b2f60 msgr2=0x7fa5d815e670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:55.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 1 -- 10.20.193.90:0/1430903331 shutdown_connections 2026-02-25T15:07:55.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 1 -- 10.20.193.90:0/1430903331 wait complete. 2026-02-25T15:07:55.740 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.735+0000 7fa5defc0640 1 librados: shutdown 2026-02-25T15:07:55.744 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 12 2026-02-25T15:07:55.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.811+0000 7f7583dfa640 10 monclient: get_monmap_and_config 2026-02-25T15:07:55.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.811+0000 7f7583dfa640 10 monclient: build_initial_monmap 2026-02-25T15:07:55.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.811+0000 7f7583dfa640 10 monclient: monmap: 2026-02-25T15:07:55.813 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:55.813 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.814 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:55.812466+0000 2026-02-25T15:07:55.814 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:55.812466+0000 2026-02-25T15:07:55.814 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:55.814 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.814 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:55.814 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:55.814 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:55.814 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.812+0000 7f7583dfa640 1 Processor -- start 2026-02-25T15:07:55.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.812+0000 7f7583dfa640 1 -- start start 2026-02-25T15:07:55.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.812+0000 7f7583dfa640 10 monclient: init 2026-02-25T15:07:55.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.812+0000 7f7583dfa640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:55.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.812+0000 7f7583dfa640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:07:55.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.812+0000 7f7583dfa640 10 monclient(hunting): picked mon.noname-b con 0x7f757c157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:55.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.812+0000 7f7583dfa640 10 monclient(hunting): picked mon.noname-a con 0x7f757c157b50 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:55.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.812+0000 7f7583dfa640 10 monclient(hunting): picked mon.noname-c con 0x7f757c164d00 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:55.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.812+0000 7f7583dfa640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757c05de70 con 0x7f757c157b50 2026-02-25T15:07:55.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.812+0000 7f7583dfa640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757c055b20 con 0x7f757c157780 2026-02-25T15:07:55.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.812+0000 7f7583dfa640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757c055e00 con 0x7f757c164d00 2026-02-25T15:07:55.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.812+0000 7f7583dfa640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:55.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.812+0000 7f7583dfa640 10 monclient(hunting): authenticate will time out at 3365.412787s 2026-02-25T15:07:55.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.813+0000 7f758136e640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f757c157b50 0x7f757c1645f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39724/0 (socket says 10.20.193.90:39724) 2026-02-25T15:07:55.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.813+0000 7f758136e640 1 -- 10.20.193.90:0/3793887405 learned_addr learned my addr 10.20.193.90:0/3793887405 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:55.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.814+0000 7f7580b6d640 1 -- 10.20.193.90:0/3793887405 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 263800784 0 0) 0x7f757c055b20 con 0x7f757c157780 2026-02-25T15:07:55.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.814+0000 7f7580b6d640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.814+0000 7f7580b6d640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.814+0000 7f7580b6d640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.814+0000 7f7580b6d640 10 monclient(hunting): my global_id is 27457 2026-02-25T15:07:55.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.814+0000 7f7580b6d640 1 -- 10.20.193.90:0/3793887405 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7564003610 con 0x7f757c157780 2026-02-25T15:07:55.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.814+0000 7f7580b6d640 1 -- 10.20.193.90:0/3793887405 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1978557435 0 0) 0x7f757c05de70 con 0x7f757c157b50 2026-02-25T15:07:55.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.814+0000 7f7580b6d640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.814+0000 7f7580b6d640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.814+0000 7f7580b6d640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.814+0000 7f7580b6d640 10 monclient(hunting): my global_id is 30825 2026-02-25T15:07:55.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.814+0000 7f7580b6d640 1 -- 10.20.193.90:0/3793887405 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f757c055b20 con 0x7f757c157b50 2026-02-25T15:07:55.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.815+0000 7f7580b6d640 1 -- 10.20.193.90:0/3793887405 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4005417140 0 0) 0x7f757c055e00 con 0x7f757c164d00 2026-02-25T15:07:55.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.815+0000 7f7580b6d640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.815+0000 7f7580b6d640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.815+0000 7f7580b6d640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.815+0000 7f7580b6d640 10 monclient(hunting): my global_id is 28595 2026-02-25T15:07:55.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.815+0000 7f7580b6d640 1 -- 10.20.193.90:0/3793887405 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f757c05de70 con 0x7f757c164d00 2026-02-25T15:07:55.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 1 -- 10.20.193.90:0/3793887405 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4047446071 0 0) 0x7f757c05de70 con 0x7f757c164d00 2026-02-25T15:07:55.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 1 -- 10.20.193.90:0/3793887405 >> v1:10.20.193.93:6789/0 conn(0x7f757c157780 legacy=0x7f757c056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 1 -- 10.20.193.90:0/3793887405 >> v1:10.20.193.90:6789/0 conn(0x7f757c157b50 legacy=0x7f757c1645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 10 monclient: _finish_hunting 0 2026-02-25T15:07:55.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 1 monclient: found mon.noname-c 2026-02-25T15:07:55.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:55.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:07:55.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 1 -- 10.20.193.90:0/3793887405 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c0560e0 con 0x7f757c164d00 2026-02-25T15:07:55.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 10 monclient: _finish_auth 0 2026-02-25T15:07:55.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 10 monclient: _check_auth_tickets 2026-02-25T15:07:55.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:55.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 1 -- 10.20.193.90:0/3793887405 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7578003150 con 0x7f757c164d00 2026-02-25T15:07:55.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:55.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:07:55.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 10 monclient: dump: 2026-02-25T15:07:55.828 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.828 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.828 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.829 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.829 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.829 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.829 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.829 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.829 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.829 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7583dfa640 5 monclient: authenticate success, global_id 28595 2026-02-25T15:07:55.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7583dfa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:55.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 1 -- 10.20.193.90:0/3793887405 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7578003320 con 0x7f757c164d00 2026-02-25T15:07:55.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:55.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 1 -- 10.20.193.90:0/3793887405 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7578003760 con 0x7f757c164d00 2026-02-25T15:07:55.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:55.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:55.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7580b6d640 10 monclient: dump: 2026-02-25T15:07:55.830 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.830 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.830 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.830 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.830 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.830 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.831 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.831 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.831 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.831 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7583dfa640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:55.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.817+0000 7f7583dfa640 4 set_mon_vals no callback set 2026-02-25T15:07:55.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 10 monclient: shutdown 2026-02-25T15:07:55.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:55.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 1 -- 10.20.193.90:0/3793887405 >> v1:10.20.193.94:6789/0 conn(0x7f757c164d00 legacy=0x7f757c1c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 1 -- 10.20.193.90:0/3793887405 shutdown_connections 2026-02-25T15:07:55.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 1 -- 10.20.193.90:0/3793887405 >> 10.20.193.90:0/3793887405 conn(0x7f757c0b2ef0 msgr2=0x7f757c0b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:55.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 1 -- 10.20.193.90:0/3793887405 shutdown_connections 2026-02-25T15:07:55.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 1 -- 10.20.193.90:0/3793887405 wait complete. 2026-02-25T15:07:55.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 10 monclient: build_initial_monmap 2026-02-25T15:07:55.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 10 monclient: monmap: 2026-02-25T15:07:55.832 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:55.832 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:55.832 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:55.832 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:55.832 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:55.832 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:55.832 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:55.833 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:55.833 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:55.833 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 1 librados: starting msgr at 2026-02-25T15:07:55.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 1 librados: starting objecter 2026-02-25T15:07:55.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:55.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 1 Processor -- start 2026-02-25T15:07:55.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 1 -- start start 2026-02-25T15:07:55.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 1 librados: setting wanted keys 2026-02-25T15:07:55.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 1 librados: calling monclient init 2026-02-25T15:07:55.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.818+0000 7f7583dfa640 10 monclient: init 2026-02-25T15:07:55.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f7583dfa640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:55.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f7583dfa640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:07:55.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f7583dfa640 10 monclient(hunting): picked mon.noname-a con 0x7f757c157780 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:55.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f7583dfa640 10 monclient(hunting): picked mon.noname-c con 0x7f757c157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:55.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f7583dfa640 10 monclient(hunting): picked mon.noname-b con 0x7f757c164d00 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:55.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f7583dfa640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757c15b990 con 0x7f757c157780 2026-02-25T15:07:55.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f7583dfa640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757c1363b0 con 0x7f757c164d00 2026-02-25T15:07:55.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f7583dfa640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757c15bc30 con 0x7f757c157b50 2026-02-25T15:07:55.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f7583dfa640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:55.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f7583dfa640 10 monclient(hunting): authenticate will time out at 3365.419454s 2026-02-25T15:07:55.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f7581b6f640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f757c157780 0x7f757c15e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39732/0 (socket says 10.20.193.90:39732) 2026-02-25T15:07:55.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f7581b6f640 1 -- 10.20.193.90:0/1487398035 learned_addr learned my addr 10.20.193.90:0/1487398035 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:55.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f756a7fc640 10 client.?.objecter ms_handle_connect 0x7f757c157780 2026-02-25T15:07:55.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f756a7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:55.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f756a7fc640 10 client.?.objecter ms_handle_connect 0x7f757c164d00 2026-02-25T15:07:55.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f756a7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:55.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f756a7fc640 10 client.?.objecter ms_handle_connect 0x7f757c157b50 2026-02-25T15:07:55.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.819+0000 7f756a7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:55.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1375773455 0 0) 0x7f757c15b990 con 0x7f757c157780 2026-02-25T15:07:55.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 10 monclient(hunting): my global_id is 30828 2026-02-25T15:07:55.835 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f754c003610 con 0x7f757c157780 2026-02-25T15:07:55.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1573101700 0 0) 0x7f757c1363b0 con 0x7f757c164d00 2026-02-25T15:07:55.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 10 monclient(hunting): my global_id is 27460 2026-02-25T15:07:55.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f757c15b990 con 0x7f757c164d00 2026-02-25T15:07:55.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2749422610 0 0) 0x7f757c15bc30 con 0x7f757c157b50 2026-02-25T15:07:55.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:55.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:55.836 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:55.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 10 monclient(hunting): my global_id is 28598 2026-02-25T15:07:55.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.820+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f757c1363b0 con 0x7f757c157b50 2026-02-25T15:07:55.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.822+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2919272431 0 0) 0x7f754c003610 con 0x7f757c157780 2026-02-25T15:07:55.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.822+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f757c15bc30 con 0x7f757c157780 2026-02-25T15:07:55.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2298886823 0 0) 0x7f757c1363b0 con 0x7f757c157b50 2026-02-25T15:07:55.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f754c003610 con 0x7f757c157b50 2026-02-25T15:07:55.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f75700030b0 con 0x7f757c157b50 2026-02-25T15:07:55.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:55.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:07:55.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 10 monclient(hunting): dump: 2026-02-25T15:07:55.838 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.838 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.838 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.838 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.838 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.838 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.838 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.838 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.838 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.838 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.838 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:55.838 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3785787006 0 0) 0x7f757c15b990 con 0x7f757c164d00 2026-02-25T15:07:55.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f757c1363b0 con 0x7f757c164d00 2026-02-25T15:07:55.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f756c003240 con 0x7f757c157780 2026-02-25T15:07:55.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:55.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:55.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 10 monclient(hunting): dump: 2026-02-25T15:07:55.839 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.839 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.839 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.839 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.839 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.839 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.840 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.840 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.840 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.840 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f758136e640 1 -- 10.20.193.90:0/1487398035 >> v1:10.20.193.94:6789/0 conn(0x7f757c157b50 legacy=0x7f757c135770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-25T15:07:55.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f758136e640 1 -- 10.20.193.90:0/1487398035 >> v1:10.20.193.94:6789/0 conn(0x7f757c157b50 legacy=0x7f757c135770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-25T15:07:55.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f758136e640 1 --1- 10.20.193.90:0/1487398035 >> v1:10.20.193.94:6789/0 conn(0x7f757c157b50 0x7f757c135770 :-1 s=OPENED pgs=9137 cs=1 l=1).handle_message read tag failed 2026-02-25T15:07:55.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f758136e640 1 --1- 10.20.193.90:0/1487398035 >> v1:10.20.193.94:6789/0 conn(0x7f757c157b50 0x7f757c135770 :-1 s=OPENED pgs=9137 cs=1 l=1).fault on lossy channel, failing 2026-02-25T15:07:55.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.94:6789/0 2026-02-25T15:07:55.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7578003670 con 0x7f757c164d00 2026-02-25T15:07:55.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:55.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:55.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.823+0000 7f756a7fc640 10 monclient(hunting): dump: 2026-02-25T15:07:55.841 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.841 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.841 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.841 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.841 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.841 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.841 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.841 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.841 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.841 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1613010849 0 0) 0x7f757c15bc30 con 0x7f757c157780 2026-02-25T15:07:55.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 >> v1:10.20.193.94:6789/0 conn(0x7f757c157b50 legacy=0x7f757c135770 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-25T15:07:55.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 >> v1:10.20.193.93:6789/0 conn(0x7f757c164d00 legacy=0x7f757c0577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:55.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f756a7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:07:55.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f756a7fc640 1 monclient: found mon.a 2026-02-25T15:07:55.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f756a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:55.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f756a7fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:55.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c15bf10 con 0x7f757c157780 2026-02-25T15:07:55.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f756a7fc640 10 monclient: _finish_auth 0 2026-02-25T15:07:55.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f756a7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:07:55.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f756a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:55.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f7583dfa640 5 monclient: authenticate success, global_id 30828 2026-02-25T15:07:55.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f7583dfa640 10 monclient: _renew_subs 2026-02-25T15:07:55.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f7583dfa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:55.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f7583dfa640 1 -- 10.20.193.90:0/1487398035 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f757c15c1f0 con 0x7f757c157780 2026-02-25T15:07:55.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f7583dfa640 10 client.30828.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:55.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f7583dfa640 10 monclient: _renew_subs 2026-02-25T15:07:55.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f7583dfa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:55.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f7583dfa640 1 -- 10.20.193.90:0/1487398035 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f757c15a9a0 con 0x7f757c157780 2026-02-25T15:07:55.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.824+0000 7f7583dfa640 1 librados: init done 2026-02-25T15:07:55.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.825+0000 7f7583dfa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:55.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.825+0000 7f7583dfa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:55.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.825+0000 7f7583dfa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:55.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.825+0000 7f7583dfa640 1 -- 10.20.193.90:0/1487398035 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f757c0531e0 con 0x7f757c157780 2026-02-25T15:07:55.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.833+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f756c0033e0 con 0x7f757c157780 2026-02-25T15:07:55.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.833+0000 7f756a7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:55.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.833+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f756c003b50 con 0x7f757c157780 2026-02-25T15:07:55.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.833+0000 7f756a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:55.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.833+0000 7f756a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:55.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.833+0000 7f756a7fc640 10 monclient: dump: 2026-02-25T15:07:55.845 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:55.846 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:55.846 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.846 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:55.846 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:55.846 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:55.846 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:55.846 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:55.846 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:55.846 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:55.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.833+0000 7f756bfff640 4 set_mon_vals no callback set 2026-02-25T15:07:55.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.833+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f756c003d30 con 0x7f757c157780 2026-02-25T15:07:55.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.834+0000 7f758136e640 10 monclient: get_auth_request con 0x7f754c07a040 auth_method 0 2026-02-25T15:07:55.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.834+0000 7f756a7fc640 10 client.30828.objecter ms_handle_connect 0x7f754c07a040 2026-02-25T15:07:55.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.835+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(337..337 src has 1..337) ==== 8959+0+0 (unknown 3556021173 0 0) 0x7f756c05acb0 con 0x7f757c157780 2026-02-25T15:07:55.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.835+0000 7f756a7fc640 10 client.30828.objecter ms_dispatch2 0x7f757c000b90 osd_map(337..337 src has 1..337) 2026-02-25T15:07:55.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.835+0000 7f756a7fc640 3 client.30828.objecter handle_osd_map got epochs [337,337] > 0 2026-02-25T15:07:55.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.835+0000 7f756a7fc640 3 client.30828.objecter handle_osd_map decoding full epoch 337 2026-02-25T15:07:55.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.835+0000 7f756a7fc640 20 client.30828.objecter dump_active .. 0 homeless 2026-02-25T15:07:55.920 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.919+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f757c0531e0 con 0x7f757c157780 2026-02-25T15:07:55.921 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.919+0000 7f756a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:55.921 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:55.919+0000 7f756a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:56.027 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.025+0000 7f7583dfa640 1 -- 10.20.193.90:0/1487398035 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "12", "target": ["mon-mgr", ""]}) -- 0x7f757c15f620 con 0x7f754c07a040 2026-02-25T15:07:56.027 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.025+0000 7f756a7fc640 1 -- 10.20.193.90:0/1487398035 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 12 to deep-scrub) ==== 42+0+0 (unknown 660425183 0 0) 0x7f757c15f620 con 0x7f754c07a040 2026-02-25T15:07:56.027 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 12 to deep-scrub 2026-02-25T15:07:56.029 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.027+0000 7f7583dfa640 10 librados: watch_flush enter 2026-02-25T15:07:56.029 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.027+0000 7f7583dfa640 10 librados: watch_flush exit 2026-02-25T15:07:56.029 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.027+0000 7f7583dfa640 20 client.30828.objecter shutdown clearing up homeless session... 2026-02-25T15:07:56.029 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.027+0000 7f7583dfa640 10 client.30828.objecter successfully canceled tick 2026-02-25T15:07:56.029 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.027+0000 7f7583dfa640 1 -- 10.20.193.90:0/1487398035 >> v1:10.20.193.189:6800/3647375982 conn(0x7f754c07a040 legacy=0x7f754c09a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.029 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.027+0000 7f7583dfa640 10 monclient: shutdown 2026-02-25T15:07:56.029 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.027+0000 7f7583dfa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:56.029 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.027+0000 7f7583dfa640 1 -- 10.20.193.90:0/1487398035 >> v1:10.20.193.90:6789/0 conn(0x7f757c157780 legacy=0x7f757c15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.030 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.027+0000 7f7583dfa640 1 -- 10.20.193.90:0/1487398035 shutdown_connections 2026-02-25T15:07:56.030 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.027+0000 7f7583dfa640 1 -- 10.20.193.90:0/1487398035 >> 10.20.193.90:0/1487398035 conn(0x7f757c0b2ef0 msgr2=0x7f757c0b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:56.030 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.027+0000 7f7583dfa640 1 -- 10.20.193.90:0/1487398035 shutdown_connections 2026-02-25T15:07:56.030 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.027+0000 7f7583dfa640 1 -- 10.20.193.90:0/1487398035 wait complete. 2026-02-25T15:07:56.030 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.027+0000 7f7583dfa640 1 librados: shutdown 2026-02-25T15:07:56.038 INFO:tasks.ceph:Scrubbing osd.1 2026-02-25T15:07:56.039 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:07:56.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.106+0000 7fab43964640 10 monclient: get_monmap_and_config 2026-02-25T15:07:56.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.106+0000 7fab43964640 10 monclient: build_initial_monmap 2026-02-25T15:07:56.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.106+0000 7fab43964640 10 monclient: monmap: 2026-02-25T15:07:56.109 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:56.109 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.109 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:56.107204+0000 2026-02-25T15:07:56.109 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:56.107204+0000 2026-02-25T15:07:56.109 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:56.109 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.109 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:56.109 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:56.109 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:56.109 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.106+0000 7fab43964640 1 Processor -- start 2026-02-25T15:07:56.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.107+0000 7fab43964640 1 -- start start 2026-02-25T15:07:56.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.107+0000 7fab43964640 10 monclient: init 2026-02-25T15:07:56.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.107+0000 7fab43964640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:56.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.107+0000 7fab43964640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:07:56.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.107+0000 7fab43964640 10 monclient(hunting): picked mon.noname-b con 0x7fab3c15e2a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:56.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.107+0000 7fab43964640 10 monclient(hunting): picked mon.noname-a con 0x7fab3c059e10 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:56.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.107+0000 7fab43964640 10 monclient(hunting): picked mon.noname-c con 0x7fab3c1a5660 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:56.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.107+0000 7fab43964640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c162df0 con 0x7fab3c059e10 2026-02-25T15:07:56.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.107+0000 7fab43964640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c1630d0 con 0x7fab3c15e2a0 2026-02-25T15:07:56.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.107+0000 7fab43964640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c1633b0 con 0x7fab3c1a5660 2026-02-25T15:07:56.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.107+0000 7fab43964640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:56.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.107+0000 7fab43964640 10 monclient(hunting): authenticate will time out at 3365.707574s 2026-02-25T15:07:56.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.107+0000 7fab416d9640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fab3c15e2a0 0x7fab3c05da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45558/0 (socket says 10.20.193.90:45558) 2026-02-25T15:07:56.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.107+0000 7fab416d9640 1 -- 10.20.193.90:0/3044087602 learned_addr learned my addr 10.20.193.90:0/3044087602 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:56.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 1 -- 10.20.193.90:0/3044087602 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2832536432 0 0) 0x7fab3c1633b0 con 0x7fab3c1a5660 2026-02-25T15:07:56.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 10 monclient(hunting): my global_id is 28601 2026-02-25T15:07:56.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 1 -- 10.20.193.90:0/3044087602 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab24003610 con 0x7fab3c1a5660 2026-02-25T15:07:56.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 1 -- 10.20.193.90:0/3044087602 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4035144493 0 0) 0x7fab3c162df0 con 0x7fab3c059e10 2026-02-25T15:07:56.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 10 monclient(hunting): my global_id is 30831 2026-02-25T15:07:56.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 1 -- 10.20.193.90:0/3044087602 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab3c1633b0 con 0x7fab3c059e10 2026-02-25T15:07:56.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 1 -- 10.20.193.90:0/3044087602 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2529536232 0 0) 0x7fab3c1630d0 con 0x7fab3c15e2a0 2026-02-25T15:07:56.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 10 monclient(hunting): my global_id is 27463 2026-02-25T15:07:56.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.109+0000 7fab2bfff640 1 -- 10.20.193.90:0/3044087602 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab3c162df0 con 0x7fab3c15e2a0 2026-02-25T15:07:56.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab2bfff640 1 -- 10.20.193.90:0/3044087602 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1205431345 0 0) 0x7fab24003610 con 0x7fab3c1a5660 2026-02-25T15:07:56.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab2bfff640 1 -- 10.20.193.90:0/3044087602 >> v1:10.20.193.93:6789/0 conn(0x7fab3c15e2a0 legacy=0x7fab3c05da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab2bfff640 1 -- 10.20.193.90:0/3044087602 >> v1:10.20.193.90:6789/0 conn(0x7fab3c059e10 legacy=0x7fab3c15ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab2bfff640 10 monclient: _finish_hunting 0 2026-02-25T15:07:56.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab2bfff640 1 monclient: found mon.noname-c 2026-02-25T15:07:56.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:56.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab2bfff640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:07:56.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab2bfff640 1 -- 10.20.193.90:0/3044087602 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3c163690 con 0x7fab3c1a5660 2026-02-25T15:07:56.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab2bfff640 10 monclient: _finish_auth 0 2026-02-25T15:07:56.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab2bfff640 10 monclient: _check_auth_tickets 2026-02-25T15:07:56.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab2bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:56.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab2bfff640 1 -- 10.20.193.90:0/3044087602 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab38003170 con 0x7fab3c1a5660 2026-02-25T15:07:56.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab43964640 5 monclient: authenticate success, global_id 28601 2026-02-25T15:07:56.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab43964640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:56.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab2bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:56.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab2bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:07:56.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab2bfff640 10 monclient: dump: 2026-02-25T15:07:56.121 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.121 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.121 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.121 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.122 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.122 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.122 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.122 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.122 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.122 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.110+0000 7fab43964640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:56.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.111+0000 7fab2bfff640 1 -- 10.20.193.90:0/3044087602 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fab38003340 con 0x7fab3c1a5660 2026-02-25T15:07:56.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.111+0000 7fab2bfff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:56.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.111+0000 7fab2bfff640 1 -- 10.20.193.90:0/3044087602 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab38003780 con 0x7fab3c1a5660 2026-02-25T15:07:56.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.111+0000 7fab43964640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:56.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.111+0000 7fab43964640 4 set_mon_vals no callback set 2026-02-25T15:07:56.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.111+0000 7fab2bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:56.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.111+0000 7fab2bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:56.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.111+0000 7fab2bfff640 10 monclient: dump: 2026-02-25T15:07:56.123 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.123 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.123 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.123 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.123 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.123 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.124 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.124 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.124 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.124 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 10 monclient: shutdown 2026-02-25T15:07:56.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:56.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 1 -- 10.20.193.90:0/3044087602 >> v1:10.20.193.94:6789/0 conn(0x7fab3c1a5660 legacy=0x7fab3c1c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 1 -- 10.20.193.90:0/3044087602 shutdown_connections 2026-02-25T15:07:56.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 1 -- 10.20.193.90:0/3044087602 >> 10.20.193.90:0/3044087602 conn(0x7fab3c0b2f60 msgr2=0x7fab3c0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:56.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 1 -- 10.20.193.90:0/3044087602 shutdown_connections 2026-02-25T15:07:56.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 1 -- 10.20.193.90:0/3044087602 wait complete. 2026-02-25T15:07:56.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 10 monclient: build_initial_monmap 2026-02-25T15:07:56.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 10 monclient: monmap: 2026-02-25T15:07:56.125 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:56.125 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:56.125 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:56.125 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:56.125 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:56.125 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:56.125 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:56.125 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:56.126 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:56.126 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 1 librados: starting msgr at 2026-02-25T15:07:56.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 1 librados: starting objecter 2026-02-25T15:07:56.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:56.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 1 Processor -- start 2026-02-25T15:07:56.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 1 -- start start 2026-02-25T15:07:56.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 1 librados: setting wanted keys 2026-02-25T15:07:56.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 1 librados: calling monclient init 2026-02-25T15:07:56.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.112+0000 7fab43964640 10 monclient: init 2026-02-25T15:07:56.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab43964640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:56.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab43964640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:07:56.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab43964640 10 monclient(hunting): picked mon.noname-a con 0x7fab3c059e10 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:56.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab43964640 10 monclient(hunting): picked mon.noname-c con 0x7fab3c15e2a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:56.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab43964640 10 monclient(hunting): picked mon.noname-b con 0x7fab3c1a5660 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:56.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab43964640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c0ad080 con 0x7fab3c059e10 2026-02-25T15:07:56.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab43964640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c05be90 con 0x7fab3c1a5660 2026-02-25T15:07:56.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab43964640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c05c170 con 0x7fab3c15e2a0 2026-02-25T15:07:56.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab43964640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:56.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab43964640 10 monclient(hunting): authenticate will time out at 3365.713413s 2026-02-25T15:07:56.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab416d9640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fab3c059e10 0x7fab3c0ac970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39744/0 (socket says 10.20.193.90:39744) 2026-02-25T15:07:56.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab416d9640 1 -- 10.20.193.90:0/2120548898 learned_addr learned my addr 10.20.193.90:0/2120548898 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:56.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab29ffb640 10 client.?.objecter ms_handle_connect 0x7fab3c059e10 2026-02-25T15:07:56.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab29ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:56.128 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab29ffb640 10 client.?.objecter ms_handle_connect 0x7fab3c1a5660 2026-02-25T15:07:56.128 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab29ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:56.128 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab29ffb640 10 client.?.objecter ms_handle_connect 0x7fab3c15e2a0 2026-02-25T15:07:56.128 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.113+0000 7fab29ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:56.128 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2652743309 0 0) 0x7fab3c0ad080 con 0x7fab3c059e10 2026-02-25T15:07:56.128 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.128 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.128 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.128 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 10 monclient(hunting): my global_id is 30834 2026-02-25T15:07:56.128 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab0c003610 con 0x7fab3c059e10 2026-02-25T15:07:56.128 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 195512474 0 0) 0x7fab3c05c170 con 0x7fab3c15e2a0 2026-02-25T15:07:56.128 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 10 monclient(hunting): my global_id is 28604 2026-02-25T15:07:56.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab3c0ad080 con 0x7fab3c15e2a0 2026-02-25T15:07:56.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2627512426 0 0) 0x7fab3c05be90 con 0x7fab3c1a5660 2026-02-25T15:07:56.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 10 monclient(hunting): my global_id is 27466 2026-02-25T15:07:56.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.114+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab3c05c170 con 0x7fab3c1a5660 2026-02-25T15:07:56.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2957586467 0 0) 0x7fab3c05c170 con 0x7fab3c1a5660 2026-02-25T15:07:56.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fab3c05be90 con 0x7fab3c1a5660 2026-02-25T15:07:56.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab380034f0 con 0x7fab3c1a5660 2026-02-25T15:07:56.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:56.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:56.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 10 monclient(hunting): dump: 2026-02-25T15:07:56.134 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.134 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.135 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.135 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.135 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.135 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.135 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.135 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.135 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.135 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:56.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3244166799 0 0) 0x7fab3c0ad080 con 0x7fab3c15e2a0 2026-02-25T15:07:56.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fab3c05c170 con 0x7fab3c15e2a0 2026-02-25T15:07:56.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 175426814 0 0) 0x7fab0c003610 con 0x7fab3c059e10 2026-02-25T15:07:56.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fab3c0ad080 con 0x7fab3c059e10 2026-02-25T15:07:56.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab30002f70 con 0x7fab3c15e2a0 2026-02-25T15:07:56.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:56.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:56.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 10 monclient(hunting): dump: 2026-02-25T15:07:56.136 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.136 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.136 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.136 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.136 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.136 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.137 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.137 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.137 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.137 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab2c002fe0 con 0x7fab3c059e10 2026-02-25T15:07:56.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:56.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:56.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.117+0000 7fab29ffb640 10 monclient(hunting): dump: 2026-02-25T15:07:56.137 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.137 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.137 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.137 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.137 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.138 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.138 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.138 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.138 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.138 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3885005396 0 0) 0x7fab3c05be90 con 0x7fab3c1a5660 2026-02-25T15:07:56.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 >> v1:10.20.193.94:6789/0 conn(0x7fab3c15e2a0 legacy=0x7fab3c12bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 >> v1:10.20.193.90:6789/0 conn(0x7fab3c059e10 legacy=0x7fab3c0ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab29ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:07:56.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab29ffb640 1 monclient: found mon.b 2026-02-25T15:07:56.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab29ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:56.138 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab29ffb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:56.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3c05c450 con 0x7fab3c1a5660 2026-02-25T15:07:56.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab29ffb640 10 monclient: _finish_auth 0 2026-02-25T15:07:56.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab29ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:07:56.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab29ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:56.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab43964640 5 monclient: authenticate success, global_id 27466 2026-02-25T15:07:56.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab43964640 10 monclient: _renew_subs 2026-02-25T15:07:56.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab43964640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:56.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab43964640 1 -- 10.20.193.90:0/2120548898 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fab3c05c730 con 0x7fab3c1a5660 2026-02-25T15:07:56.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab43964640 10 client.27466.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:56.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab43964640 10 monclient: _renew_subs 2026-02-25T15:07:56.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab43964640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:56.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab43964640 1 -- 10.20.193.90:0/2120548898 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fab3c05ccf0 con 0x7fab3c1a5660 2026-02-25T15:07:56.139 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.118+0000 7fab43964640 1 librados: init done 2026-02-25T15:07:56.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.119+0000 7fab43964640 10 client.27466.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:56.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.119+0000 7fab43964640 15 client.27466.objecter _session_command_op_assign -1 1 2026-02-25T15:07:56.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.119+0000 7fab43964640 20 client.27466.objecter _get_session osd=-1 returning homeless 2026-02-25T15:07:56.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.119+0000 7fab43964640 10 client.27466.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:56.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.119+0000 7fab43964640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:56.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.119+0000 7fab43964640 1 -- 10.20.193.90:0/2120548898 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fab3c160320 con 0x7fab3c1a5660 2026-02-25T15:07:56.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.128+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fab3c160320 con 0x7fab3c1a5660 2026-02-25T15:07:56.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.128+0000 7fab29ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:56.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.128+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fab38003300 con 0x7fab3c1a5660 2026-02-25T15:07:56.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.128+0000 7fab29ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:56.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.128+0000 7fab29ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:56.140 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.128+0000 7fab2b7fe640 4 set_mon_vals no callback set 2026-02-25T15:07:56.141 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.128+0000 7fab29ffb640 10 monclient: dump: 2026-02-25T15:07:56.141 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.141 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.141 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.141 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.141 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.141 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.141 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.141 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.141 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.141 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.141 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.129+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fab3801d020 con 0x7fab3c1a5660 2026-02-25T15:07:56.141 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.129+0000 7fab416d9640 10 monclient: get_auth_request con 0x7fab0c07a170 auth_method 0 2026-02-25T15:07:56.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.129+0000 7fab29ffb640 10 client.27466.objecter ms_handle_connect 0x7fab0c07a170 2026-02-25T15:07:56.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(337..337 src has 1..337) ==== 8959+0+0 (unknown 3556021173 0 0) 0x7fab38094a30 con 0x7fab3c1a5660 2026-02-25T15:07:56.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 10 client.27466.objecter ms_dispatch2 0x7fab3c000b90 osd_map(337..337 src has 1..337) 2026-02-25T15:07:56.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 3 client.27466.objecter handle_osd_map got epochs [337,337] > 0 2026-02-25T15:07:56.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 3 client.27466.objecter handle_osd_map decoding full epoch 337 2026-02-25T15:07:56.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 10 client.27466.objecter checking command 1 2026-02-25T15:07:56.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 20 client.27466.objecter _get_session s=0x7fab0c0a4420 osd=1 3 2026-02-25T15:07:56.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 20 client.27466.objecter put_session s=0x7fab0c0a4420 osd=1 3 2026-02-25T15:07:56.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 15 client.27466.objecter _session_command_op_remove -1 1 2026-02-25T15:07:56.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 20 client.27466.objecter _get_session s=0x7fab0c0a4420 osd=1 3 2026-02-25T15:07:56.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 20 client.27466.objecter get_session s=0x7fab0c0a4420 osd=1 3 2026-02-25T15:07:56.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 15 client.27466.objecter _session_command_op_assign 1 1 2026-02-25T15:07:56.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 20 client.27466.objecter put_session s=0x7fab0c0a4420 osd=1 4 2026-02-25T15:07:56.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 10 client.27466.objecter _send_command 1 2026-02-25T15:07:56.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 --> v1:10.20.193.93:6804/3161035071 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab38003690 con 0x7fab0c0a4ad0 2026-02-25T15:07:56.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 20 client.27466.objecter dump_active .. 0 homeless 2026-02-25T15:07:56.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_get_version_reply(handle=1 version=337) ==== 24+0+0 (unknown 676316437 0 0) 0x7fab3805a000 con 0x7fab3c1a5660 2026-02-25T15:07:56.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab29ffb640 10 monclient: handle_get_version_reply finishing 1 version 337 2026-02-25T15:07:56.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.130+0000 7fab40ed8640 10 monclient: get_auth_request con 0x7fab0c0a4ad0 auth_method 0 2026-02-25T15:07:56.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.131+0000 7fab29ffb640 10 client.27466.objecter ms_handle_connect 0x7fab0c0a4ad0 2026-02-25T15:07:56.144 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.140+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== osd.1 v1:10.20.193.93:6804/3161035071 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7fab38003690 con 0x7fab0c0a4ad0 2026-02-25T15:07:56.144 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.140+0000 7fab29ffb640 10 client.27466.objecter ms_dispatch2 0x7fab3c000b90 command_reply(tid 1: 0 ) 2026-02-25T15:07:56.144 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.140+0000 7fab29ffb640 10 client.27466.objecter _finish_command 1 = system:0 2026-02-25T15:07:56.144 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.140+0000 7fab29ffb640 20 client.27466.objecter put_session s=0x7fab0c0a4420 osd=1 4 2026-02-25T15:07:56.144 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.140+0000 7fab29ffb640 15 client.27466.objecter _session_command_op_remove 1 1 2026-02-25T15:07:56.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.147+0000 7fab43964640 10 client.27466.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:07:56.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.147+0000 7fab43964640 15 client.27466.objecter _session_command_op_assign -1 2 2026-02-25T15:07:56.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.147+0000 7fab43964640 20 client.27466.objecter _get_session s=0x7fab0c0a4420 osd=1 3 2026-02-25T15:07:56.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.147+0000 7fab43964640 20 client.27466.objecter put_session s=0x7fab0c0a4420 osd=1 3 2026-02-25T15:07:56.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.147+0000 7fab43964640 20 client.27466.objecter _get_session s=0x7fab0c0a4420 osd=1 3 2026-02-25T15:07:56.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.147+0000 7fab43964640 15 client.27466.objecter _session_command_op_remove -1 2 2026-02-25T15:07:56.149 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.147+0000 7fab43964640 20 client.27466.objecter get_session s=0x7fab0c0a4420 osd=1 3 2026-02-25T15:07:56.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.147+0000 7fab43964640 15 client.27466.objecter _session_command_op_assign 1 2 2026-02-25T15:07:56.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.147+0000 7fab43964640 20 client.27466.objecter put_session s=0x7fab0c0a4420 osd=1 4 2026-02-25T15:07:56.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.147+0000 7fab43964640 10 client.27466.objecter _send_command 2 2026-02-25T15:07:56.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.147+0000 7fab43964640 1 -- 10.20.193.90:0/2120548898 --> v1:10.20.193.93:6804/3161035071 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fab3c05e110 con 0x7fab0c0a4ad0 2026-02-25T15:07:56.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab29ffb640 1 -- 10.20.193.90:0/2120548898 <== osd.1 v1:10.20.193.93:6804/3161035071 2 ==== command_reply(tid 2: 0 ) ==== 8+0+712 (unknown 0 0 714369635) 0x7fab3c05e110 con 0x7fab0c0a4ad0 2026-02-25T15:07:56.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab29ffb640 10 client.27466.objecter ms_dispatch2 0x7fab3c000b90 command_reply(tid 2: 0 ) 2026-02-25T15:07:56.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab29ffb640 10 client.27466.objecter _finish_command 2 = system:0 2026-02-25T15:07:56.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab29ffb640 20 client.27466.objecter put_session s=0x7fab0c0a4420 osd=1 4 2026-02-25T15:07:56.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab29ffb640 15 client.27466.objecter _session_command_op_remove 1 2 2026-02-25T15:07:56.164 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:07:56.164 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-25T15:07:56.165 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:07:56.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 10 librados: watch_flush enter 2026-02-25T15:07:56.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 10 librados: watch_flush exit 2026-02-25T15:07:56.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 10 client.27466.objecter close_session for osd.1 2026-02-25T15:07:56.166 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 1 -- 10.20.193.90:0/2120548898 >> v1:10.20.193.93:6804/3161035071 conn(0x7fab0c0a4ad0 legacy=0x7fab0c0c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 20 client.27466.objecter put_session s=0x7fab0c0a4420 osd=1 1 2026-02-25T15:07:56.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 20 client.27466.objecter shutdown clearing up homeless session... 2026-02-25T15:07:56.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 10 client.27466.objecter successfully canceled tick 2026-02-25T15:07:56.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 1 -- 10.20.193.90:0/2120548898 >> v1:10.20.193.189:6800/3647375982 conn(0x7fab0c07a170 legacy=0x7fab0c09a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 10 monclient: shutdown 2026-02-25T15:07:56.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:56.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 1 -- 10.20.193.90:0/2120548898 >> v1:10.20.193.93:6789/0 conn(0x7fab3c1a5660 legacy=0x7fab3c0a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab41eda640 1 -- 10.20.193.90:0/2120548898 reap_dead start 2026-02-25T15:07:56.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 1 -- 10.20.193.90:0/2120548898 shutdown_connections 2026-02-25T15:07:56.167 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 1 -- 10.20.193.90:0/2120548898 >> 10.20.193.90:0/2120548898 conn(0x7fab3c0b2f60 msgr2=0x7fab3c161f30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:56.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 1 -- 10.20.193.90:0/2120548898 shutdown_connections 2026-02-25T15:07:56.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 1 -- 10.20.193.90:0/2120548898 wait complete. 2026-02-25T15:07:56.168 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.162+0000 7fab43964640 1 librados: shutdown 2026-02-25T15:07:56.171 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-25T15:07:56.239 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.238+0000 7f32f768a640 10 monclient: get_monmap_and_config 2026-02-25T15:07:56.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.238+0000 7f32f768a640 10 monclient: build_initial_monmap 2026-02-25T15:07:56.240 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.238+0000 7f32f768a640 10 monclient: monmap: 2026-02-25T15:07:56.240 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:56.240 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.240 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:56.238925+0000 2026-02-25T15:07:56.240 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:56.238925+0000 2026-02-25T15:07:56.240 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:56.240 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.240 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:56.240 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:56.240 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:56.241 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.238+0000 7f32f768a640 1 Processor -- start 2026-02-25T15:07:56.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.238+0000 7f32f768a640 1 -- start start 2026-02-25T15:07:56.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.238+0000 7f32f768a640 10 monclient: init 2026-02-25T15:07:56.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.239+0000 7f32f768a640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:56.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.239+0000 7f32f768a640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:07:56.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.239+0000 7f32f768a640 10 monclient(hunting): picked mon.noname-a con 0x7f32f01612c0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:56.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.239+0000 7f32f768a640 10 monclient(hunting): picked mon.noname-c con 0x7f32f00b58e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:56.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.239+0000 7f32f768a640 10 monclient(hunting): picked mon.noname-b con 0x7f32f01620d0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:56.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.239+0000 7f32f768a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32f005b7d0 con 0x7f32f01612c0 2026-02-25T15:07:56.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.239+0000 7f32f768a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32f005bab0 con 0x7f32f01620d0 2026-02-25T15:07:56.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.239+0000 7f32f768a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32f005bd90 con 0x7f32f00b58e0 2026-02-25T15:07:56.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.239+0000 7f32f768a640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:56.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.239+0000 7f32f768a640 10 monclient(hunting): authenticate will time out at 3365.839320s 2026-02-25T15:07:56.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.239+0000 7f32f6688640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f32f01612c0 0x7f32f00b51d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39754/0 (socket says 10.20.193.90:39754) 2026-02-25T15:07:56.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.239+0000 7f32f6688640 1 -- 10.20.193.90:0/985678901 learned_addr learned my addr 10.20.193.90:0/985678901 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:56.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.240+0000 7f32f5686640 1 -- 10.20.193.90:0/985678901 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2160601841 0 0) 0x7f32f005b7d0 con 0x7f32f01612c0 2026-02-25T15:07:56.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.240+0000 7f32f5686640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.240+0000 7f32f5686640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.240+0000 7f32f5686640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.240+0000 7f32f5686640 10 monclient(hunting): my global_id is 30837 2026-02-25T15:07:56.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.241+0000 7f32f5686640 1 -- 10.20.193.90:0/985678901 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32e0003610 con 0x7f32f01612c0 2026-02-25T15:07:56.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.241+0000 7f32f5686640 1 -- 10.20.193.90:0/985678901 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 493112642 0 0) 0x7f32f005bd90 con 0x7f32f00b58e0 2026-02-25T15:07:56.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.241+0000 7f32f5686640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.241+0000 7f32f5686640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.241+0000 7f32f5686640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.241+0000 7f32f5686640 10 monclient(hunting): my global_id is 28607 2026-02-25T15:07:56.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.241+0000 7f32f5686640 1 -- 10.20.193.90:0/985678901 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32f005b7d0 con 0x7f32f00b58e0 2026-02-25T15:07:56.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.241+0000 7f32f5686640 1 -- 10.20.193.90:0/985678901 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3778515866 0 0) 0x7f32f005bab0 con 0x7f32f01620d0 2026-02-25T15:07:56.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.241+0000 7f32f5686640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.241+0000 7f32f5686640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.241+0000 7f32f5686640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.241+0000 7f32f5686640 10 monclient(hunting): my global_id is 27469 2026-02-25T15:07:56.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.241+0000 7f32f5686640 1 -- 10.20.193.90:0/985678901 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32f005bd90 con 0x7f32f01620d0 2026-02-25T15:07:56.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f5686640 1 -- 10.20.193.90:0/985678901 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 256266742 0 0) 0x7f32e0003610 con 0x7f32f01612c0 2026-02-25T15:07:56.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f5686640 1 -- 10.20.193.90:0/985678901 >> v1:10.20.193.94:6789/0 conn(0x7f32f00b58e0 legacy=0x7f32f0055980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f5686640 1 -- 10.20.193.90:0/985678901 >> v1:10.20.193.93:6789/0 conn(0x7f32f01620d0 legacy=0x7f32f0056090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f5686640 10 monclient: _finish_hunting 0 2026-02-25T15:07:56.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f5686640 1 monclient: found mon.noname-a 2026-02-25T15:07:56.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f5686640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:56.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f5686640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:07:56.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f5686640 1 -- 10.20.193.90:0/985678901 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f005c070 con 0x7f32f01612c0 2026-02-25T15:07:56.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f5686640 10 monclient: _finish_auth 0 2026-02-25T15:07:56.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f5686640 10 monclient: _check_auth_tickets 2026-02-25T15:07:56.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f5686640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:56.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f5686640 1 -- 10.20.193.90:0/985678901 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f32e8002df0 con 0x7f32f01612c0 2026-02-25T15:07:56.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f5686640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:56.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f5686640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:56.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f5686640 10 monclient: dump: 2026-02-25T15:07:56.254 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.254 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.254 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.254 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.254 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.254 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.254 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.254 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.254 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.254 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f768a640 5 monclient: authenticate success, global_id 30837 2026-02-25T15:07:56.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.242+0000 7f32f768a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:56.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.243+0000 7f32f5686640 1 -- 10.20.193.90:0/985678901 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f32e8003030 con 0x7f32f01612c0 2026-02-25T15:07:56.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.243+0000 7f32f5686640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:56.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.243+0000 7f32f5686640 1 -- 10.20.193.90:0/985678901 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f32e8003470 con 0x7f32f01612c0 2026-02-25T15:07:56.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.243+0000 7f32f5686640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:56.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.243+0000 7f32f5686640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:56.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.243+0000 7f32f5686640 10 monclient: dump: 2026-02-25T15:07:56.255 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.255 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.255 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.255 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.256 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.256 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.256 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.256 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.256 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.256 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.243+0000 7f32f768a640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:56.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.243+0000 7f32f768a640 4 set_mon_vals no callback set 2026-02-25T15:07:56.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.243+0000 7f32f768a640 10 monclient: shutdown 2026-02-25T15:07:56.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.243+0000 7f32f768a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:56.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.243+0000 7f32f768a640 1 -- 10.20.193.90:0/985678901 >> v1:10.20.193.90:6789/0 conn(0x7f32f01612c0 legacy=0x7f32f00b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.243+0000 7f32f768a640 1 -- 10.20.193.90:0/985678901 shutdown_connections 2026-02-25T15:07:56.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.243+0000 7f32f768a640 1 -- 10.20.193.90:0/985678901 >> 10.20.193.90:0/985678901 conn(0x7f32f00b2d40 msgr2=0x7f32f00b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:56.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.243+0000 7f32f768a640 1 -- 10.20.193.90:0/985678901 shutdown_connections 2026-02-25T15:07:56.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.243+0000 7f32f768a640 1 -- 10.20.193.90:0/985678901 wait complete. 2026-02-25T15:07:56.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.244+0000 7f32f768a640 10 monclient: build_initial_monmap 2026-02-25T15:07:56.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.244+0000 7f32f768a640 10 monclient: monmap: 2026-02-25T15:07:56.257 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:56.257 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:56.257 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:56.257 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:56.257 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:56.257 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:56.257 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:56.258 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:56.258 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:56.258 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.244+0000 7f32f768a640 1 librados: starting msgr at 2026-02-25T15:07:56.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.244+0000 7f32f768a640 1 librados: starting objecter 2026-02-25T15:07:56.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.244+0000 7f32f768a640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:56.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.244+0000 7f32f768a640 1 Processor -- start 2026-02-25T15:07:56.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.244+0000 7f32f768a640 1 -- start start 2026-02-25T15:07:56.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.244+0000 7f32f768a640 1 librados: setting wanted keys 2026-02-25T15:07:56.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.244+0000 7f32f768a640 1 librados: calling monclient init 2026-02-25T15:07:56.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.244+0000 7f32f768a640 10 monclient: init 2026-02-25T15:07:56.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.244+0000 7f32f768a640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:56.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.244+0000 7f32f768a640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:07:56.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.245+0000 7f32f768a640 10 monclient(hunting): picked mon.noname-b con 0x7f32f00b58e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:56.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.245+0000 7f32f768a640 10 monclient(hunting): picked mon.noname-a con 0x7f32f01612c0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:56.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.245+0000 7f32f768a640 10 monclient(hunting): picked mon.noname-c con 0x7f32f01620d0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:56.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.245+0000 7f32f768a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32f0147010 con 0x7f32f01612c0 2026-02-25T15:07:56.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.245+0000 7f32f768a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32f00b2780 con 0x7f32f00b58e0 2026-02-25T15:07:56.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.245+0000 7f32f768a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32f00588b0 con 0x7f32f01620d0 2026-02-25T15:07:56.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.245+0000 7f32f768a640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:56.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.245+0000 7f32f768a640 10 monclient(hunting): authenticate will time out at 3365.845036s 2026-02-25T15:07:56.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.245+0000 7f32f5e87640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f32f01612c0 0x7f32f00b2070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39770/0 (socket says 10.20.193.90:39770) 2026-02-25T15:07:56.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.245+0000 7f32f5e87640 1 -- 10.20.193.90:0/2746522399 learned_addr learned my addr 10.20.193.90:0/2746522399 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:56.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.245+0000 7f32daffd640 10 client.?.objecter ms_handle_connect 0x7f32f01612c0 2026-02-25T15:07:56.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.245+0000 7f32daffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:56.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.245+0000 7f32daffd640 10 client.?.objecter ms_handle_connect 0x7f32f00b58e0 2026-02-25T15:07:56.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.245+0000 7f32daffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:56.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.246+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 777559455 0 0) 0x7f32f0147010 con 0x7f32f01612c0 2026-02-25T15:07:56.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.246+0000 7f32daffd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.246+0000 7f32daffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.246+0000 7f32daffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.246+0000 7f32daffd640 10 monclient(hunting): my global_id is 30840 2026-02-25T15:07:56.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.246+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32c0003610 con 0x7f32f01612c0 2026-02-25T15:07:56.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.246+0000 7f32daffd640 10 client.?.objecter ms_handle_connect 0x7f32f01620d0 2026-02-25T15:07:56.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.246+0000 7f32daffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:56.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.246+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3498662280 0 0) 0x7f32f00b2780 con 0x7f32f00b58e0 2026-02-25T15:07:56.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.246+0000 7f32daffd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.246+0000 7f32daffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.246+0000 7f32daffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.246+0000 7f32daffd640 10 monclient(hunting): my global_id is 27472 2026-02-25T15:07:56.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.246+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32f0147010 con 0x7f32f00b58e0 2026-02-25T15:07:56.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.247+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2650255701 0 0) 0x7f32f00588b0 con 0x7f32f01620d0 2026-02-25T15:07:56.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.247+0000 7f32daffd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.247+0000 7f32daffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.247+0000 7f32daffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.247+0000 7f32daffd640 10 monclient(hunting): my global_id is 28610 2026-02-25T15:07:56.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.247+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32f00b2780 con 0x7f32f01620d0 2026-02-25T15:07:56.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.248+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3594253498 0 0) 0x7f32c0003610 con 0x7f32f01612c0 2026-02-25T15:07:56.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.248+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f32f00588b0 con 0x7f32f01612c0 2026-02-25T15:07:56.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.248+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f32dc0030f0 con 0x7f32f01612c0 2026-02-25T15:07:56.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.248+0000 7f32daffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:56.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.248+0000 7f32daffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:56.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.248+0000 7f32daffd640 10 monclient(hunting): dump: 2026-02-25T15:07:56.262 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.262 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.262 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.262 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.262 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.262 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.262 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.262 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.263 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.263 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.248+0000 7f32daffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:56.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.249+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4242922764 0 0) 0x7f32f0147010 con 0x7f32f00b58e0 2026-02-25T15:07:56.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.249+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f32c0003610 con 0x7f32f00b58e0 2026-02-25T15:07:56.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.249+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f32e80031e0 con 0x7f32f00b58e0 2026-02-25T15:07:56.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.249+0000 7f32daffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:56.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.249+0000 7f32daffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:56.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.249+0000 7f32daffd640 10 monclient(hunting): dump: 2026-02-25T15:07:56.263 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.263 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.263 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.264 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.264 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.264 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.264 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.264 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.264 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.264 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2324688428 0 0) 0x7f32f00b2780 con 0x7f32f01620d0 2026-02-25T15:07:56.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f32f0147010 con 0x7f32f01620d0 2026-02-25T15:07:56.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f32e40033e0 con 0x7f32f01620d0 2026-02-25T15:07:56.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:56.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:56.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 10 monclient(hunting): dump: 2026-02-25T15:07:56.265 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.265 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.265 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.265 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.265 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.265 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.265 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.265 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.265 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.265 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3739061661 0 0) 0x7f32f00588b0 con 0x7f32f01612c0 2026-02-25T15:07:56.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 >> v1:10.20.193.94:6789/0 conn(0x7f32f01620d0 legacy=0x7f32f0145740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 >> v1:10.20.193.93:6789/0 conn(0x7f32f00b58e0 legacy=0x7f32f0146900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 10 monclient: _finish_hunting 0 2026-02-25T15:07:56.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 1 monclient: found mon.a 2026-02-25T15:07:56.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:56.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:56.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f0058b90 con 0x7f32f01612c0 2026-02-25T15:07:56.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 10 monclient: _finish_auth 0 2026-02-25T15:07:56.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 10 monclient: _check_auth_tickets 2026-02-25T15:07:56.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:56.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32f768a640 5 monclient: authenticate success, global_id 30840 2026-02-25T15:07:56.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32f768a640 10 monclient: _renew_subs 2026-02-25T15:07:56.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32f768a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:56.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32f768a640 1 -- 10.20.193.90:0/2746522399 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f32f0058e70 con 0x7f32f01612c0 2026-02-25T15:07:56.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32f768a640 10 client.30840.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:56.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32f768a640 10 monclient: _renew_subs 2026-02-25T15:07:56.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32f768a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:56.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32f768a640 1 -- 10.20.193.90:0/2746522399 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f32f0059430 con 0x7f32f01612c0 2026-02-25T15:07:56.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32f768a640 1 librados: init done 2026-02-25T15:07:56.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32f768a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:56.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32f768a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:56.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32f768a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:56.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.250+0000 7f32f768a640 1 -- 10.20.193.90:0/2746522399 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f0163c10 con 0x7f32f01612c0 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.340+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f32dc003440 con 0x7f32f01612c0 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.340+0000 7f32daffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.340+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f32dc003b50 con 0x7f32f01612c0 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.340+0000 7f32daffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.340+0000 7f32daffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.340+0000 7f32daffd640 10 monclient: dump: 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.340+0000 7f32f4e85640 4 set_mon_vals no callback set 2026-02-25T15:07:56.342 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.341+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f32dc003d30 con 0x7f32f01612c0 2026-02-25T15:07:56.343 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.342+0000 7f32f6688640 10 monclient: get_auth_request con 0x7f32c007a090 auth_method 0 2026-02-25T15:07:56.343 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.342+0000 7f32daffd640 10 client.30840.objecter ms_handle_connect 0x7f32c007a090 2026-02-25T15:07:56.347 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.345+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(337..337 src has 1..337) ==== 8959+0+0 (unknown 3556021173 0 0) 0x7f32dc095a00 con 0x7f32f01612c0 2026-02-25T15:07:56.347 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.345+0000 7f32daffd640 10 client.30840.objecter ms_dispatch2 0x7f32f0000b90 osd_map(337..337 src has 1..337) 2026-02-25T15:07:56.347 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.345+0000 7f32daffd640 3 client.30840.objecter handle_osd_map got epochs [337,337] > 0 2026-02-25T15:07:56.347 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.345+0000 7f32daffd640 3 client.30840.objecter handle_osd_map decoding full epoch 337 2026-02-25T15:07:56.347 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.345+0000 7f32daffd640 20 client.30840.objecter dump_active .. 0 homeless 2026-02-25T15:07:56.347 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.345+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f32dc05f360 con 0x7f32f01612c0 2026-02-25T15:07:56.347 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.345+0000 7f32daffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:56.347 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.345+0000 7f32daffd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:56.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.452+0000 7f32f768a640 1 -- 10.20.193.90:0/2746522399 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f32f0163930 con 0x7f32c007a090 2026-02-25T15:07:56.454 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.453+0000 7f32daffd640 1 -- 10.20.193.90:0/2746522399 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (unknown 489605429 0 0) 0x7f32f0163930 con 0x7f32c007a090 2026-02-25T15:07:56.455 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 1 to deep-scrub 2026-02-25T15:07:56.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.455+0000 7f32f768a640 10 librados: watch_flush enter 2026-02-25T15:07:56.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.455+0000 7f32f768a640 10 librados: watch_flush exit 2026-02-25T15:07:56.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.455+0000 7f32f768a640 20 client.30840.objecter shutdown clearing up homeless session... 2026-02-25T15:07:56.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.455+0000 7f32f768a640 10 client.30840.objecter successfully canceled tick 2026-02-25T15:07:56.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.455+0000 7f32f768a640 1 -- 10.20.193.90:0/2746522399 >> v1:10.20.193.189:6800/3647375982 conn(0x7f32c007a090 legacy=0x7f32c009a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.457 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.455+0000 7f32f768a640 10 monclient: shutdown 2026-02-25T15:07:56.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.455+0000 7f32f768a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:56.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.455+0000 7f32f768a640 1 -- 10.20.193.90:0/2746522399 >> v1:10.20.193.90:6789/0 conn(0x7f32f01612c0 legacy=0x7f32f00b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.455+0000 7f32f768a640 1 -- 10.20.193.90:0/2746522399 shutdown_connections 2026-02-25T15:07:56.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.455+0000 7f32f768a640 1 -- 10.20.193.90:0/2746522399 >> 10.20.193.90:0/2746522399 conn(0x7f32f00b2d40 msgr2=0x7f32f0164160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:56.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.455+0000 7f32f768a640 1 -- 10.20.193.90:0/2746522399 shutdown_connections 2026-02-25T15:07:56.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.455+0000 7f32f768a640 1 -- 10.20.193.90:0/2746522399 wait complete. 2026-02-25T15:07:56.458 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.455+0000 7f32f768a640 1 librados: shutdown 2026-02-25T15:07:56.465 INFO:tasks.ceph:Scrubbing osd.5 2026-02-25T15:07:56.465 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:07:56.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.533+0000 7f6411624640 10 monclient: get_monmap_and_config 2026-02-25T15:07:56.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.533+0000 7f6411624640 10 monclient: build_initial_monmap 2026-02-25T15:07:56.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.533+0000 7f6411624640 10 monclient: monmap: 2026-02-25T15:07:56.535 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:56.536 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.536 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:56.534072+0000 2026-02-25T15:07:56.536 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:56.534072+0000 2026-02-25T15:07:56.536 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:56.536 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.536 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:56.536 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:56.536 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:56.536 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.533+0000 7f6411624640 1 Processor -- start 2026-02-25T15:07:56.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.533+0000 7f6411624640 1 -- start start 2026-02-25T15:07:56.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.533+0000 7f6411624640 10 monclient: init 2026-02-25T15:07:56.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.534+0000 7f6411624640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:56.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.534+0000 7f6411624640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:07:56.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.534+0000 7f6411624640 10 monclient(hunting): picked mon.noname-a con 0x7f640c15e2a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:56.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.534+0000 7f6411624640 10 monclient(hunting): picked mon.noname-b con 0x7f640c059e10 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:56.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.534+0000 7f6411624640 10 monclient(hunting): picked mon.noname-c con 0x7f640c1a5660 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:56.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.534+0000 7f6411624640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f640c162df0 con 0x7f640c15e2a0 2026-02-25T15:07:56.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.534+0000 7f6411624640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f640c1630d0 con 0x7f640c059e10 2026-02-25T15:07:56.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.534+0000 7f6411624640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f640c1633b0 con 0x7f640c1a5660 2026-02-25T15:07:56.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.534+0000 7f6411624640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:56.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.534+0000 7f6411624640 10 monclient(hunting): authenticate will time out at 3366.134450s 2026-02-25T15:07:56.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.534+0000 7f640a7fc640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f640c059e10 0x7f640c15ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45598/0 (socket says 10.20.193.90:45598) 2026-02-25T15:07:56.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.534+0000 7f640a7fc640 1 -- 10.20.193.90:0/1566839005 learned_addr learned my addr 10.20.193.90:0/1566839005 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:56.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 1 -- 10.20.193.90:0/1566839005 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2810778510 0 0) 0x7f640c1633b0 con 0x7f640c1a5660 2026-02-25T15:07:56.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 10 monclient(hunting): my global_id is 28613 2026-02-25T15:07:56.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 1 -- 10.20.193.90:0/1566839005 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63f8003610 con 0x7f640c1a5660 2026-02-25T15:07:56.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 1 -- 10.20.193.90:0/1566839005 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 7641756 0 0) 0x7f640c1630d0 con 0x7f640c059e10 2026-02-25T15:07:56.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 10 monclient(hunting): my global_id is 27475 2026-02-25T15:07:56.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 1 -- 10.20.193.90:0/1566839005 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f640c1633b0 con 0x7f640c059e10 2026-02-25T15:07:56.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 1 -- 10.20.193.90:0/1566839005 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2440479869 0 0) 0x7f640c162df0 con 0x7f640c15e2a0 2026-02-25T15:07:56.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 10 monclient(hunting): my global_id is 30843 2026-02-25T15:07:56.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.536+0000 7f6409ffb640 1 -- 10.20.193.90:0/1566839005 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f640c1630d0 con 0x7f640c15e2a0 2026-02-25T15:07:56.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6409ffb640 1 -- 10.20.193.90:0/1566839005 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3539926983 0 0) 0x7f640c1630d0 con 0x7f640c15e2a0 2026-02-25T15:07:56.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6409ffb640 1 -- 10.20.193.90:0/1566839005 >> v1:10.20.193.94:6789/0 conn(0x7f640c1a5660 legacy=0x7f640c1c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6409ffb640 1 -- 10.20.193.90:0/1566839005 >> v1:10.20.193.93:6789/0 conn(0x7f640c059e10 legacy=0x7f640c15ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6409ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:07:56.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6409ffb640 1 monclient: found mon.noname-a 2026-02-25T15:07:56.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6409ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:56.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6409ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:07:56.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6409ffb640 1 -- 10.20.193.90:0/1566839005 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c163690 con 0x7f640c15e2a0 2026-02-25T15:07:56.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6409ffb640 10 monclient: _finish_auth 0 2026-02-25T15:07:56.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6409ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:07:56.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6409ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:56.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6409ffb640 1 -- 10.20.193.90:0/1566839005 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6400002dd0 con 0x7f640c15e2a0 2026-02-25T15:07:56.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6409ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:56.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6409ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:56.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6409ffb640 10 monclient: dump: 2026-02-25T15:07:56.550 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.550 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.550 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.550 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.551 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.551 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.551 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.551 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.551 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.551 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6411624640 5 monclient: authenticate success, global_id 30843 2026-02-25T15:07:56.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.538+0000 7f6411624640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:56.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.539+0000 7f6409ffb640 1 -- 10.20.193.90:0/1566839005 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6400002fa0 con 0x7f640c15e2a0 2026-02-25T15:07:56.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.539+0000 7f6409ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:56.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.539+0000 7f6409ffb640 1 -- 10.20.193.90:0/1566839005 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f64000033e0 con 0x7f640c15e2a0 2026-02-25T15:07:56.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.539+0000 7f6411624640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:56.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.539+0000 7f6411624640 4 set_mon_vals no callback set 2026-02-25T15:07:56.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.539+0000 7f6409ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:56.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.539+0000 7f6409ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:56.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.539+0000 7f6409ffb640 10 monclient: dump: 2026-02-25T15:07:56.552 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.552 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.552 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.552 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.552 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.552 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.553 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.553 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.553 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.553 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.539+0000 7f6411624640 10 monclient: shutdown 2026-02-25T15:07:56.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.539+0000 7f6411624640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:56.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.539+0000 7f6411624640 1 -- 10.20.193.90:0/1566839005 >> v1:10.20.193.90:6789/0 conn(0x7f640c15e2a0 legacy=0x7f640c05da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.539+0000 7f6411624640 1 -- 10.20.193.90:0/1566839005 shutdown_connections 2026-02-25T15:07:56.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.539+0000 7f6411624640 1 -- 10.20.193.90:0/1566839005 >> 10.20.193.90:0/1566839005 conn(0x7f640c0b2f60 msgr2=0x7f640c0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:56.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.539+0000 7f6411624640 1 -- 10.20.193.90:0/1566839005 shutdown_connections 2026-02-25T15:07:56.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.539+0000 7f6411624640 1 -- 10.20.193.90:0/1566839005 wait complete. 2026-02-25T15:07:56.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 10 monclient: build_initial_monmap 2026-02-25T15:07:56.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 10 monclient: monmap: 2026-02-25T15:07:56.554 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:56.554 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:56.554 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:56.554 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:56.554 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:56.554 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:56.554 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:56.554 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:56.554 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:56.554 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 1 librados: starting msgr at 2026-02-25T15:07:56.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 1 librados: starting objecter 2026-02-25T15:07:56.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:56.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 1 Processor -- start 2026-02-25T15:07:56.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 1 -- start start 2026-02-25T15:07:56.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 1 librados: setting wanted keys 2026-02-25T15:07:56.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 1 librados: calling monclient init 2026-02-25T15:07:56.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 10 monclient: init 2026-02-25T15:07:56.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:56.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:07:56.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 10 monclient(hunting): picked mon.noname-a con 0x7f640c059e10 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:56.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 10 monclient(hunting): picked mon.noname-b con 0x7f640c15e2a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:56.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 10 monclient(hunting): picked mon.noname-c con 0x7f640c1a5660 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:56.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f640c190820 con 0x7f640c059e10 2026-02-25T15:07:56.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f640c190b00 con 0x7f640c15e2a0 2026-02-25T15:07:56.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f640c190de0 con 0x7f640c1a5660 2026-02-25T15:07:56.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:56.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.540+0000 7f6411624640 10 monclient(hunting): authenticate will time out at 3366.140948s 2026-02-25T15:07:56.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.541+0000 7f640affd640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f640c059e10 0x7f640c11eac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39792/0 (socket says 10.20.193.90:39792) 2026-02-25T15:07:56.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.541+0000 7f640affd640 1 -- 10.20.193.90:0/3628301695 learned_addr learned my addr 10.20.193.90:0/3628301695 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:56.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.541+0000 7f63eb7fe640 10 client.?.objecter ms_handle_connect 0x7f640c059e10 2026-02-25T15:07:56.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.541+0000 7f63eb7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:56.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.541+0000 7f63eb7fe640 10 client.?.objecter ms_handle_connect 0x7f640c1a5660 2026-02-25T15:07:56.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.541+0000 7f63eb7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:56.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.541+0000 7f63eb7fe640 10 client.?.objecter ms_handle_connect 0x7f640c15e2a0 2026-02-25T15:07:56.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.541+0000 7f63eb7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:56.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4165239766 0 0) 0x7f640c190820 con 0x7f640c059e10 2026-02-25T15:07:56.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 10 monclient(hunting): my global_id is 30846 2026-02-25T15:07:56.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63d8003610 con 0x7f640c059e10 2026-02-25T15:07:56.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1655377238 0 0) 0x7f640c190de0 con 0x7f640c1a5660 2026-02-25T15:07:56.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 10 monclient(hunting): my global_id is 28616 2026-02-25T15:07:56.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f640c190820 con 0x7f640c1a5660 2026-02-25T15:07:56.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 591539142 0 0) 0x7f640c190b00 con 0x7f640c15e2a0 2026-02-25T15:07:56.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 10 monclient(hunting): my global_id is 27478 2026-02-25T15:07:56.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.542+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f640c190de0 con 0x7f640c15e2a0 2026-02-25T15:07:56.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.544+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2685722348 0 0) 0x7f640c190820 con 0x7f640c1a5660 2026-02-25T15:07:56.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.544+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f640c190b00 con 0x7f640c1a5660 2026-02-25T15:07:56.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.544+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f63fc003430 con 0x7f640c1a5660 2026-02-25T15:07:56.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.544+0000 7f63eb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:56.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.544+0000 7f63eb7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:07:56.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.544+0000 7f63eb7fe640 10 monclient(hunting): dump: 2026-02-25T15:07:56.559 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.559 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.559 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.559 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.559 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.559 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.559 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.559 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.560 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.560 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.544+0000 7f63eb7fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:56.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.545+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 420070374 0 0) 0x7f63d8003610 con 0x7f640c059e10 2026-02-25T15:07:56.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.545+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f640c190820 con 0x7f640c059e10 2026-02-25T15:07:56.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.545+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f64000033b0 con 0x7f640c059e10 2026-02-25T15:07:56.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.545+0000 7f63eb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:56.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.545+0000 7f63eb7fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:56.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.545+0000 7f63eb7fe640 10 monclient(hunting): dump: 2026-02-25T15:07:56.560 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.560 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.561 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.561 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.561 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.561 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.561 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.561 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.561 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.561 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.545+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3603062927 0 0) 0x7f640c190de0 con 0x7f640c15e2a0 2026-02-25T15:07:56.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.545+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f63d8003610 con 0x7f640c15e2a0 2026-02-25T15:07:56.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.545+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f63f4002f30 con 0x7f640c15e2a0 2026-02-25T15:07:56.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.545+0000 7f63eb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:56.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.545+0000 7f63eb7fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:56.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.545+0000 7f63eb7fe640 10 monclient(hunting): dump: 2026-02-25T15:07:56.562 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.562 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.562 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.562 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.562 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.562 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.562 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.562 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.562 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.562 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 162641039 0 0) 0x7f640c190b00 con 0x7f640c1a5660 2026-02-25T15:07:56.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 >> v1:10.20.193.93:6789/0 conn(0x7f640c15e2a0 legacy=0x7f640c15d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 >> v1:10.20.193.90:6789/0 conn(0x7f640c059e10 legacy=0x7f640c11eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f63eb7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:07:56.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f63eb7fe640 1 monclient: found mon.c 2026-02-25T15:07:56.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f63eb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:56.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f63eb7fe640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:56.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c1910c0 con 0x7f640c1a5660 2026-02-25T15:07:56.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f63eb7fe640 10 monclient: _finish_auth 0 2026-02-25T15:07:56.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f63eb7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:07:56.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f63eb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:56.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f6411624640 5 monclient: authenticate success, global_id 28616 2026-02-25T15:07:56.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f6411624640 10 monclient: _renew_subs 2026-02-25T15:07:56.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f6411624640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:56.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f6411624640 1 -- 10.20.193.90:0/3628301695 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f640c1913a0 con 0x7f640c1a5660 2026-02-25T15:07:56.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f6411624640 10 client.28616.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:56.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f6411624640 10 monclient: _renew_subs 2026-02-25T15:07:56.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f6411624640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:56.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f6411624640 1 -- 10.20.193.90:0/3628301695 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f640c15d930 con 0x7f640c1a5660 2026-02-25T15:07:56.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f6411624640 1 librados: init done 2026-02-25T15:07:56.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f6411624640 10 client.28616.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:56.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f6411624640 15 client.28616.objecter _session_command_op_assign -1 1 2026-02-25T15:07:56.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f6411624640 20 client.28616.objecter _get_session osd=-1 returning homeless 2026-02-25T15:07:56.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f6411624640 10 client.28616.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:56.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f6411624640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:07:56.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.546+0000 7f6411624640 1 -- 10.20.193.90:0/3628301695 --> v1:10.20.193.94:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f640c0a9830 con 0x7f640c1a5660 2026-02-25T15:07:56.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.554+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f640c0a9830 con 0x7f640c1a5660 2026-02-25T15:07:56.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.554+0000 7f63eb7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:56.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.554+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f63fc003b50 con 0x7f640c1a5660 2026-02-25T15:07:56.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.554+0000 7f63eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:56.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.554+0000 7f63eb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:56.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.554+0000 7f64097fa640 4 set_mon_vals no callback set 2026-02-25T15:07:56.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.554+0000 7f63eb7fe640 10 monclient: dump: 2026-02-25T15:07:56.567 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.567 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.567 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.567 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.567 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.567 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.567 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.567 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.567 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.567 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.555+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f63fc01d030 con 0x7f640c1a5660 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.555+0000 7f640affd640 10 monclient: get_auth_request con 0x7f63d807a060 auth_method 0 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 10 client.28616.objecter ms_handle_connect 0x7f63d807a060 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7f63fc094a00 con 0x7f640c1a5660 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 10 client.28616.objecter ms_dispatch2 0x7f640c000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 3 client.28616.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 3 client.28616.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 10 client.28616.objecter checking command 1 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 20 client.28616.objecter _get_session s=0x7f63d80a4370 osd=5 3 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 20 client.28616.objecter put_session s=0x7f63d80a4370 osd=5 3 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 15 client.28616.objecter _session_command_op_remove -1 1 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 20 client.28616.objecter _get_session s=0x7f63d80a4370 osd=5 3 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 20 client.28616.objecter get_session s=0x7f63d80a4370 osd=5 3 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 15 client.28616.objecter _session_command_op_assign 5 1 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 20 client.28616.objecter put_session s=0x7f63d80a4370 osd=5 4 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 10 client.28616.objecter _send_command 1 2026-02-25T15:07:56.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 --> v1:10.20.193.93:6800/887686167 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63d80c6680 con 0x7f63d80a4a20 2026-02-25T15:07:56.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.556+0000 7f63eb7fe640 20 client.28616.objecter dump_active .. 0 homeless 2026-02-25T15:07:56.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.557+0000 7f640a7fc640 10 monclient: get_auth_request con 0x7f63d80a4a20 auth_method 0 2026-02-25T15:07:56.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.557+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_get_version_reply(handle=1 version=338) ==== 24+0+0 (unknown 1001891302 0 0) 0x7f63fc05da30 con 0x7f640c1a5660 2026-02-25T15:07:56.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.557+0000 7f63eb7fe640 10 monclient: handle_get_version_reply finishing 1 version 338 2026-02-25T15:07:56.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.558+0000 7f63eb7fe640 10 client.28616.objecter ms_handle_connect 0x7f63d80a4a20 2026-02-25T15:07:56.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.567+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== osd.5 v1:10.20.193.93:6800/887686167 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f63d80c6680 con 0x7f63d80a4a20 2026-02-25T15:07:56.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.567+0000 7f63eb7fe640 10 client.28616.objecter ms_dispatch2 0x7f640c000b90 command_reply(tid 1: 0 ) 2026-02-25T15:07:56.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.567+0000 7f63eb7fe640 10 client.28616.objecter _finish_command 1 = system:0 2026-02-25T15:07:56.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.567+0000 7f63eb7fe640 20 client.28616.objecter put_session s=0x7f63d80a4370 osd=5 4 2026-02-25T15:07:56.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.567+0000 7f63eb7fe640 15 client.28616.objecter _session_command_op_remove 5 1 2026-02-25T15:07:56.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.574+0000 7f6411624640 10 client.28616.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:07:56.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.574+0000 7f6411624640 15 client.28616.objecter _session_command_op_assign -1 2 2026-02-25T15:07:56.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.574+0000 7f6411624640 20 client.28616.objecter _get_session s=0x7f63d80a4370 osd=5 3 2026-02-25T15:07:56.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.574+0000 7f6411624640 20 client.28616.objecter put_session s=0x7f63d80a4370 osd=5 3 2026-02-25T15:07:56.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.574+0000 7f6411624640 20 client.28616.objecter _get_session s=0x7f63d80a4370 osd=5 3 2026-02-25T15:07:56.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.574+0000 7f6411624640 15 client.28616.objecter _session_command_op_remove -1 2 2026-02-25T15:07:56.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.574+0000 7f6411624640 20 client.28616.objecter get_session s=0x7f63d80a4370 osd=5 3 2026-02-25T15:07:56.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.574+0000 7f6411624640 15 client.28616.objecter _session_command_op_assign 5 2 2026-02-25T15:07:56.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.574+0000 7f6411624640 20 client.28616.objecter put_session s=0x7f63d80a4370 osd=5 4 2026-02-25T15:07:56.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.574+0000 7f6411624640 10 client.28616.objecter _send_command 2 2026-02-25T15:07:56.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.574+0000 7f6411624640 1 -- 10.20.193.90:0/3628301695 --> v1:10.20.193.93:6800/887686167 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f640c0ad060 con 0x7f63d80a4a20 2026-02-25T15:07:56.583 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f63eb7fe640 1 -- 10.20.193.90:0/3628301695 <== osd.5 v1:10.20.193.93:6800/887686167 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f640c0ad060 con 0x7f63d80a4a20 2026-02-25T15:07:56.583 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f63eb7fe640 10 client.28616.objecter ms_dispatch2 0x7f640c000b90 command_reply(tid 2: 0 ) 2026-02-25T15:07:56.583 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f63eb7fe640 10 client.28616.objecter _finish_command 2 = system:0 2026-02-25T15:07:56.583 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f63eb7fe640 20 client.28616.objecter put_session s=0x7f63d80a4370 osd=5 4 2026-02-25T15:07:56.583 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f63eb7fe640 15 client.28616.objecter _session_command_op_remove 5 2 2026-02-25T15:07:56.583 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:07:56.583 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-25T15:07:56.584 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:07:56.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 10 librados: watch_flush enter 2026-02-25T15:07:56.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 10 librados: watch_flush exit 2026-02-25T15:07:56.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 10 client.28616.objecter close_session for osd.5 2026-02-25T15:07:56.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 1 -- 10.20.193.90:0/3628301695 >> v1:10.20.193.93:6800/887686167 conn(0x7f63d80a4a20 legacy=0x7f63d80c4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 20 client.28616.objecter put_session s=0x7f63d80a4370 osd=5 1 2026-02-25T15:07:56.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 20 client.28616.objecter shutdown clearing up homeless session... 2026-02-25T15:07:56.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 10 client.28616.objecter successfully canceled tick 2026-02-25T15:07:56.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 1 -- 10.20.193.90:0/3628301695 >> v1:10.20.193.189:6800/3647375982 conn(0x7f63d807a060 legacy=0x7f63d809a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 10 monclient: shutdown 2026-02-25T15:07:56.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:56.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 1 -- 10.20.193.90:0/3628301695 >> v1:10.20.193.94:6789/0 conn(0x7f640c1a5660 legacy=0x7f640c18fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f640b7fe640 1 -- 10.20.193.90:0/3628301695 reap_dead start 2026-02-25T15:07:56.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 1 -- 10.20.193.90:0/3628301695 shutdown_connections 2026-02-25T15:07:56.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 1 -- 10.20.193.90:0/3628301695 >> 10.20.193.90:0/3628301695 conn(0x7f640c0b2f60 msgr2=0x7f640c0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:56.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 1 -- 10.20.193.90:0/3628301695 shutdown_connections 2026-02-25T15:07:56.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 1 -- 10.20.193.90:0/3628301695 wait complete. 2026-02-25T15:07:56.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.581+0000 7f6411624640 1 librados: shutdown 2026-02-25T15:07:56.591 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2026-02-25T15:07:56.660 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.658+0000 7fb43d733640 10 monclient: get_monmap_and_config 2026-02-25T15:07:56.660 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.658+0000 7fb43d733640 10 monclient: build_initial_monmap 2026-02-25T15:07:56.660 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.658+0000 7fb43d733640 10 monclient: monmap: 2026-02-25T15:07:56.660 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:56.661 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.661 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:56.659401+0000 2026-02-25T15:07:56.661 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:56.659401+0000 2026-02-25T15:07:56.661 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:56.661 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.661 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:56.661 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:56.661 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:56.661 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.664 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.659+0000 7fb43d733640 1 Processor -- start 2026-02-25T15:07:56.664 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.659+0000 7fb43d733640 1 -- start start 2026-02-25T15:07:56.664 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.659+0000 7fb43d733640 10 monclient: init 2026-02-25T15:07:56.664 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.659+0000 7fb43d733640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:56.664 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.659+0000 7fb43d733640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:07:56.664 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.659+0000 7fb43d733640 10 monclient(hunting): picked mon.noname-b con 0x7fb438157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:56.664 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.659+0000 7fb43d733640 10 monclient(hunting): picked mon.noname-c con 0x7fb438157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:56.664 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.659+0000 7fb43d733640 10 monclient(hunting): picked mon.noname-a con 0x7fb438164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:56.665 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.659+0000 7fb43d733640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb43805de70 con 0x7fb438164d00 2026-02-25T15:07:56.665 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.659+0000 7fb43d733640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb438055b20 con 0x7fb438157780 2026-02-25T15:07:56.665 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.659+0000 7fb43d733640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb438055e00 con 0x7fb438157b50 2026-02-25T15:07:56.665 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.659+0000 7fb43d733640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:56.665 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.659+0000 7fb43d733640 10 monclient(hunting): authenticate will time out at 3366.259820s 2026-02-25T15:07:56.665 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.660+0000 7fb4377fe640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fb438164d00 0x7fb4381c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39800/0 (socket says 10.20.193.90:39800) 2026-02-25T15:07:56.665 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.660+0000 7fb4377fe640 1 -- 10.20.193.90:0/3519720820 learned_addr learned my addr 10.20.193.90:0/3519720820 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:56.665 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 1 -- 10.20.193.90:0/3519720820 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3378902859 0 0) 0x7fb438055e00 con 0x7fb438157b50 2026-02-25T15:07:56.665 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.665 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.665 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 10 monclient(hunting): my global_id is 28619 2026-02-25T15:07:56.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 1 -- 10.20.193.90:0/3519720820 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb424003610 con 0x7fb438157b50 2026-02-25T15:07:56.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 1 -- 10.20.193.90:0/3519720820 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 745356827 0 0) 0x7fb43805de70 con 0x7fb438164d00 2026-02-25T15:07:56.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 10 monclient(hunting): my global_id is 30849 2026-02-25T15:07:56.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 1 -- 10.20.193.90:0/3519720820 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb438055e00 con 0x7fb438164d00 2026-02-25T15:07:56.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 1 -- 10.20.193.90:0/3519720820 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3692584018 0 0) 0x7fb438055b20 con 0x7fb438157780 2026-02-25T15:07:56.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 10 monclient(hunting): my global_id is 27481 2026-02-25T15:07:56.667 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.661+0000 7fb435ffb640 1 -- 10.20.193.90:0/3519720820 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb43805de70 con 0x7fb438157780 2026-02-25T15:07:56.673 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 1 -- 10.20.193.90:0/3519720820 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3216119513 0 0) 0x7fb424003610 con 0x7fb438157b50 2026-02-25T15:07:56.673 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 1 -- 10.20.193.90:0/3519720820 >> v1:10.20.193.93:6789/0 conn(0x7fb438157780 legacy=0x7fb438056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.673 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 1 -- 10.20.193.90:0/3519720820 >> v1:10.20.193.90:6789/0 conn(0x7fb438164d00 legacy=0x7fb4381c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.673 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:07:56.673 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 1 monclient: found mon.noname-c 2026-02-25T15:07:56.674 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:56.674 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:07:56.674 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 1 -- 10.20.193.90:0/3519720820 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4380560e0 con 0x7fb438157b50 2026-02-25T15:07:56.674 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 10 monclient: _finish_auth 0 2026-02-25T15:07:56.674 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:07:56.674 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:56.674 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 1 -- 10.20.193.90:0/3519720820 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb420002df0 con 0x7fb438157b50 2026-02-25T15:07:56.674 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:56.674 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:07:56.674 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 10 monclient: dump: 2026-02-25T15:07:56.674 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.674 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.675 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.675 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.675 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.675 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.675 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.675 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.675 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.675 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.675 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb43d733640 5 monclient: authenticate success, global_id 28619 2026-02-25T15:07:56.675 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb43d733640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:56.675 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 1 -- 10.20.193.90:0/3519720820 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb420002fc0 con 0x7fb438157b50 2026-02-25T15:07:56.675 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:56.675 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 1 -- 10.20.193.90:0/3519720820 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb420003400 con 0x7fb438157b50 2026-02-25T15:07:56.675 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:56.676 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:56.676 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb435ffb640 10 monclient: dump: 2026-02-25T15:07:56.676 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.676 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.676 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.676 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.676 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.676 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.676 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.676 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.676 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.676 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.676 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb43d733640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:56.676 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.663+0000 7fb43d733640 4 set_mon_vals no callback set 2026-02-25T15:07:56.677 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.664+0000 7fb43d733640 10 monclient: shutdown 2026-02-25T15:07:56.677 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.664+0000 7fb43d733640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:56.677 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.664+0000 7fb43d733640 1 -- 10.20.193.90:0/3519720820 >> v1:10.20.193.94:6789/0 conn(0x7fb438157b50 legacy=0x7fb4381645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.677 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.664+0000 7fb43d733640 1 -- 10.20.193.90:0/3519720820 shutdown_connections 2026-02-25T15:07:56.677 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.664+0000 7fb43d733640 1 -- 10.20.193.90:0/3519720820 >> 10.20.193.90:0/3519720820 conn(0x7fb4380b2ef0 msgr2=0x7fb4380b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:56.677 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.664+0000 7fb43d733640 1 -- 10.20.193.90:0/3519720820 shutdown_connections 2026-02-25T15:07:56.677 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.664+0000 7fb43d733640 1 -- 10.20.193.90:0/3519720820 wait complete. 2026-02-25T15:07:56.677 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.664+0000 7fb43d733640 10 monclient: build_initial_monmap 2026-02-25T15:07:56.677 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.664+0000 7fb43d733640 10 monclient: monmap: 2026-02-25T15:07:56.677 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:56.677 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:56.677 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:56.677 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 1 librados: starting msgr at 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 1 librados: starting objecter 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 1 Processor -- start 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 1 -- start start 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 1 librados: setting wanted keys 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 1 librados: calling monclient init 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 10 monclient: init 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:56.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 10 monclient(hunting): picked mon.noname-b con 0x7fb438157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 10 monclient(hunting): picked mon.noname-c con 0x7fb438157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 10 monclient(hunting): picked mon.noname-a con 0x7fb438164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb43805d240 con 0x7fb438164d00 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4381360b0 con 0x7fb438157780 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb438136370 con 0x7fb438157b50 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb43d733640 10 monclient(hunting): authenticate will time out at 3366.265476s 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb4377fe640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fb438164d00 0x7fb43805cb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39812/0 (socket says 10.20.193.90:39812) 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb4377fe640 1 -- 10.20.193.90:0/1929700295 learned_addr learned my addr 10.20.193.90:0/1929700295 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb4177fe640 10 client.?.objecter ms_handle_connect 0x7fb438164d00 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb4177fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb4177fe640 10 client.?.objecter ms_handle_connect 0x7fb438157b50 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.665+0000 7fb4177fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.666+0000 7fb4177fe640 10 client.?.objecter ms_handle_connect 0x7fb438157780 2026-02-25T15:07:56.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.666+0000 7fb4177fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.666+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3935484909 0 0) 0x7fb43805d240 con 0x7fb438164d00 2026-02-25T15:07:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.666+0000 7fb4177fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.666+0000 7fb4177fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.666+0000 7fb4177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.666+0000 7fb4177fe640 10 monclient(hunting): my global_id is 30852 2026-02-25T15:07:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.666+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb404003610 con 0x7fb438164d00 2026-02-25T15:07:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.666+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 445857504 0 0) 0x7fb438136370 con 0x7fb438157b50 2026-02-25T15:07:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.666+0000 7fb4177fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.666+0000 7fb4177fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.666+0000 7fb4177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.666+0000 7fb4177fe640 10 monclient(hunting): my global_id is 28622 2026-02-25T15:07:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.666+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb43805d240 con 0x7fb438157b50 2026-02-25T15:07:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.667+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1615856178 0 0) 0x7fb4381360b0 con 0x7fb438157780 2026-02-25T15:07:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.667+0000 7fb4177fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.667+0000 7fb4177fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.667+0000 7fb4177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.667+0000 7fb4177fe640 10 monclient(hunting): my global_id is 27484 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.667+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb438136370 con 0x7fb438157780 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2283800282 0 0) 0x7fb404003610 con 0x7fb438164d00 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb4381360b0 con 0x7fb438164d00 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb4280032f0 con 0x7fb438164d00 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 10 monclient(hunting): dump: 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.681 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.682 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.682 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.682 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.682 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:56.682 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2226822122 0 0) 0x7fb43805d240 con 0x7fb438157b50 2026-02-25T15:07:56.682 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb404003610 con 0x7fb438157b50 2026-02-25T15:07:56.682 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb420003310 con 0x7fb438157b50 2026-02-25T15:07:56.682 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:56.682 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:56.682 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 10 monclient(hunting): dump: 2026-02-25T15:07:56.682 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.682 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.682 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.682 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.682 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.683 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.683 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.683 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.683 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.683 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3243556886 0 0) 0x7fb438136370 con 0x7fb438157780 2026-02-25T15:07:56.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb43805d240 con 0x7fb438157780 2026-02-25T15:07:56.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb42c003060 con 0x7fb438157780 2026-02-25T15:07:56.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:56.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:56.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.669+0000 7fb4177fe640 10 monclient(hunting): dump: 2026-02-25T15:07:56.683 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.683 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.683 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.683 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3161205209 0 0) 0x7fb4381360b0 con 0x7fb438164d00 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 >> v1:10.20.193.94:6789/0 conn(0x7fb438157b50 legacy=0x7fb4381359a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 >> v1:10.20.193.93:6789/0 conn(0x7fb438157780 legacy=0x7fb43813aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb4177fe640 10 monclient: _finish_hunting 0 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb4177fe640 1 monclient: found mon.a 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb4177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb4177fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb438136650 con 0x7fb438164d00 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb4177fe640 10 monclient: _finish_auth 0 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb4177fe640 10 monclient: _check_auth_tickets 2026-02-25T15:07:56.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb4177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:56.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb43d733640 5 monclient: authenticate success, global_id 30852 2026-02-25T15:07:56.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb43d733640 10 monclient: _renew_subs 2026-02-25T15:07:56.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb43d733640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:56.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb43d733640 1 -- 10.20.193.90:0/1929700295 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb438136930 con 0x7fb438164d00 2026-02-25T15:07:56.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb43d733640 10 client.30852.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:56.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb43d733640 10 monclient: _renew_subs 2026-02-25T15:07:56.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb43d733640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:56.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb43d733640 1 -- 10.20.193.90:0/1929700295 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb438136ef0 con 0x7fb438164d00 2026-02-25T15:07:56.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.670+0000 7fb43d733640 1 librados: init done 2026-02-25T15:07:56.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.671+0000 7fb43d733640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:56.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.671+0000 7fb43d733640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:56.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.671+0000 7fb43d733640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:56.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.671+0000 7fb43d733640 1 -- 10.20.193.90:0/1929700295 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb43805b740 con 0x7fb438164d00 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.763+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb428003490 con 0x7fb438164d00 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.763+0000 7fb4177fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.763+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb428004df0 con 0x7fb438164d00 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.763+0000 7fb4177fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.763+0000 7fb4177fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.763+0000 7fb4177fe640 10 monclient: dump: 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.765 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.763+0000 7fb4357fa640 4 set_mon_vals no callback set 2026-02-25T15:07:56.766 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.764+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fb428004fd0 con 0x7fb438164d00 2026-02-25T15:07:56.766 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.765+0000 7fb436ffd640 10 monclient: get_auth_request con 0x7fb404079ff0 auth_method 0 2026-02-25T15:07:56.766 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.765+0000 7fb4177fe640 10 client.30852.objecter ms_handle_connect 0x7fb404079ff0 2026-02-25T15:07:56.768 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.767+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7fb428095d80 con 0x7fb438164d00 2026-02-25T15:07:56.768 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.767+0000 7fb4177fe640 10 client.30852.objecter ms_dispatch2 0x7fb438000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:56.768 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.767+0000 7fb4177fe640 3 client.30852.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:56.768 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.767+0000 7fb4177fe640 3 client.30852.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:56.768 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.767+0000 7fb4177fe640 20 client.30852.objecter dump_active .. 0 homeless 2026-02-25T15:07:56.769 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.767+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fb428004df0 con 0x7fb438164d00 2026-02-25T15:07:56.769 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.767+0000 7fb4177fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:56.769 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.767+0000 7fb4177fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:56.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.877+0000 7fb43d733640 1 -- 10.20.193.90:0/1929700295 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fb43815ea20 con 0x7fb404079ff0 2026-02-25T15:07:56.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.877+0000 7fb4177fe640 1 -- 10.20.193.90:0/1929700295 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (unknown 2264484800 0 0) 0x7fb43815ea20 con 0x7fb404079ff0 2026-02-25T15:07:56.879 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 5 to deep-scrub 2026-02-25T15:07:56.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.879+0000 7fb43d733640 10 librados: watch_flush enter 2026-02-25T15:07:56.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.879+0000 7fb43d733640 10 librados: watch_flush exit 2026-02-25T15:07:56.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.879+0000 7fb43d733640 20 client.30852.objecter shutdown clearing up homeless session... 2026-02-25T15:07:56.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.879+0000 7fb43d733640 10 client.30852.objecter successfully canceled tick 2026-02-25T15:07:56.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.879+0000 7fb43d733640 1 -- 10.20.193.90:0/1929700295 >> v1:10.20.193.189:6800/3647375982 conn(0x7fb404079ff0 legacy=0x7fb40409a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.879+0000 7fb43d733640 10 monclient: shutdown 2026-02-25T15:07:56.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.879+0000 7fb43d733640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:56.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.879+0000 7fb43d733640 1 -- 10.20.193.90:0/1929700295 >> v1:10.20.193.90:6789/0 conn(0x7fb438164d00 legacy=0x7fb43805cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.879+0000 7fb43d733640 1 -- 10.20.193.90:0/1929700295 shutdown_connections 2026-02-25T15:07:56.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.879+0000 7fb43d733640 1 -- 10.20.193.90:0/1929700295 >> 10.20.193.90:0/1929700295 conn(0x7fb4380b2ef0 msgr2=0x7fb4380b5370 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:56.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.879+0000 7fb43d733640 1 -- 10.20.193.90:0/1929700295 shutdown_connections 2026-02-25T15:07:56.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.879+0000 7fb43d733640 1 -- 10.20.193.90:0/1929700295 wait complete. 2026-02-25T15:07:56.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.879+0000 7fb43d733640 1 librados: shutdown 2026-02-25T15:07:56.889 INFO:tasks.ceph:Scrubbing osd.9 2026-02-25T15:07:56.890 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:07:56.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.959+0000 7f6449ba9640 10 monclient: get_monmap_and_config 2026-02-25T15:07:56.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.959+0000 7f6449ba9640 10 monclient: build_initial_monmap 2026-02-25T15:07:56.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.959+0000 7f6449ba9640 10 monclient: monmap: 2026-02-25T15:07:56.961 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:56.961 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.961 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:56.960149+0000 2026-02-25T15:07:56.961 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:56.960149+0000 2026-02-25T15:07:56.961 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:56.961 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.961 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:56.961 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:56.961 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:56.961 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.959+0000 7f6449ba9640 1 Processor -- start 2026-02-25T15:07:56.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.959+0000 7f6449ba9640 1 -- start start 2026-02-25T15:07:56.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.959+0000 7f6449ba9640 10 monclient: init 2026-02-25T15:07:56.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.960+0000 7f6449ba9640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:56.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.960+0000 7f6449ba9640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:07:56.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.960+0000 7f6449ba9640 10 monclient(hunting): picked mon.noname-b con 0x7f644415e2a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:56.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.960+0000 7f6449ba9640 10 monclient(hunting): picked mon.noname-a con 0x7f6444059e10 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:56.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.960+0000 7f6449ba9640 10 monclient(hunting): picked mon.noname-c con 0x7f64441a5660 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:56.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.960+0000 7f6449ba9640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6444162df0 con 0x7f6444059e10 2026-02-25T15:07:56.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.960+0000 7f6449ba9640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64441630d0 con 0x7f644415e2a0 2026-02-25T15:07:56.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.960+0000 7f6449ba9640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64441633b0 con 0x7f64441a5660 2026-02-25T15:07:56.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.960+0000 7f6449ba9640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:56.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.960+0000 7f6449ba9640 10 monclient(hunting): authenticate will time out at 3366.560612s 2026-02-25T15:07:56.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.961+0000 7f6442ffd640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f6444059e10 0x7f644415ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39814/0 (socket says 10.20.193.90:39814) 2026-02-25T15:07:56.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.961+0000 7f6442ffd640 1 -- 10.20.193.90:0/1013008120 learned_addr learned my addr 10.20.193.90:0/1013008120 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:56.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 1 -- 10.20.193.90:0/1013008120 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4074782129 0 0) 0x7f6444162df0 con 0x7f6444059e10 2026-02-25T15:07:56.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 10 monclient(hunting): my global_id is 30855 2026-02-25T15:07:56.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 1 -- 10.20.193.90:0/1013008120 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6428003610 con 0x7f6444059e10 2026-02-25T15:07:56.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 1 -- 10.20.193.90:0/1013008120 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1763332353 0 0) 0x7f64441633b0 con 0x7f64441a5660 2026-02-25T15:07:56.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 10 monclient(hunting): my global_id is 28625 2026-02-25T15:07:56.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 1 -- 10.20.193.90:0/1013008120 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6444162df0 con 0x7f64441a5660 2026-02-25T15:07:56.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 1 -- 10.20.193.90:0/1013008120 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1095612401 0 0) 0x7f64441630d0 con 0x7f644415e2a0 2026-02-25T15:07:56.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 10 monclient(hunting): my global_id is 27487 2026-02-25T15:07:56.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.962+0000 7f64427fc640 1 -- 10.20.193.90:0/1013008120 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64441633b0 con 0x7f644415e2a0 2026-02-25T15:07:56.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f64427fc640 1 -- 10.20.193.90:0/1013008120 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1893656216 0 0) 0x7f6428003610 con 0x7f6444059e10 2026-02-25T15:07:56.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f64427fc640 1 -- 10.20.193.90:0/1013008120 >> v1:10.20.193.94:6789/0 conn(0x7f64441a5660 legacy=0x7f64441c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f64427fc640 1 -- 10.20.193.90:0/1013008120 >> v1:10.20.193.93:6789/0 conn(0x7f644415e2a0 legacy=0x7f644405da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f64427fc640 10 monclient: _finish_hunting 0 2026-02-25T15:07:56.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f64427fc640 1 monclient: found mon.noname-a 2026-02-25T15:07:56.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f64427fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:56.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f64427fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:07:56.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f64427fc640 1 -- 10.20.193.90:0/1013008120 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6444163690 con 0x7f6444059e10 2026-02-25T15:07:56.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f64427fc640 10 monclient: _finish_auth 0 2026-02-25T15:07:56.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f64427fc640 10 monclient: _check_auth_tickets 2026-02-25T15:07:56.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f64427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:56.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f64427fc640 1 -- 10.20.193.90:0/1013008120 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6434002df0 con 0x7f6444059e10 2026-02-25T15:07:56.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f64427fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:56.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f64427fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:56.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f64427fc640 10 monclient: dump: 2026-02-25T15:07:56.969 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.969 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.970 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.970 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.970 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.970 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.970 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.970 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.970 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.970 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f6449ba9640 5 monclient: authenticate success, global_id 30855 2026-02-25T15:07:56.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.964+0000 7f6449ba9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:56.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.965+0000 7f64427fc640 1 -- 10.20.193.90:0/1013008120 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6434002fc0 con 0x7f6444059e10 2026-02-25T15:07:56.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.965+0000 7f64427fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:56.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.965+0000 7f64427fc640 1 -- 10.20.193.90:0/1013008120 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6434003400 con 0x7f6444059e10 2026-02-25T15:07:56.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.965+0000 7f64427fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:56.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.965+0000 7f64427fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:56.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.965+0000 7f64427fc640 10 monclient: dump: 2026-02-25T15:07:56.971 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.971 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.971 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.971 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.971 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.971 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.971 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.971 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.972 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.972 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.972 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.965+0000 7f6449ba9640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:56.972 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.965+0000 7f6449ba9640 4 set_mon_vals no callback set 2026-02-25T15:07:56.972 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.965+0000 7f6449ba9640 10 monclient: shutdown 2026-02-25T15:07:56.972 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.965+0000 7f6449ba9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:56.972 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.965+0000 7f6449ba9640 1 -- 10.20.193.90:0/1013008120 >> v1:10.20.193.90:6789/0 conn(0x7f6444059e10 legacy=0x7f644415ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.972 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.965+0000 7f6449ba9640 1 -- 10.20.193.90:0/1013008120 shutdown_connections 2026-02-25T15:07:56.972 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.965+0000 7f6449ba9640 1 -- 10.20.193.90:0/1013008120 >> 10.20.193.90:0/1013008120 conn(0x7f64440b2f60 msgr2=0x7f64440b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:56.972 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.965+0000 7f6449ba9640 1 -- 10.20.193.90:0/1013008120 shutdown_connections 2026-02-25T15:07:56.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.965+0000 7f6449ba9640 1 -- 10.20.193.90:0/1013008120 wait complete. 2026-02-25T15:07:56.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.966+0000 7f6449ba9640 10 monclient: build_initial_monmap 2026-02-25T15:07:56.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.966+0000 7f6449ba9640 10 monclient: monmap: 2026-02-25T15:07:56.973 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:56.973 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:56.973 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:56.973 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:56.973 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:56.973 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:56.973 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:56.973 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:56.973 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:56.973 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.966+0000 7f6449ba9640 1 librados: starting msgr at 2026-02-25T15:07:56.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.966+0000 7f6449ba9640 1 librados: starting objecter 2026-02-25T15:07:56.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.966+0000 7f6449ba9640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:56.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.966+0000 7f6449ba9640 1 Processor -- start 2026-02-25T15:07:56.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.966+0000 7f6449ba9640 1 -- start start 2026-02-25T15:07:56.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.966+0000 7f6449ba9640 1 librados: setting wanted keys 2026-02-25T15:07:56.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.966+0000 7f6449ba9640 1 librados: calling monclient init 2026-02-25T15:07:56.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.966+0000 7f6449ba9640 10 monclient: init 2026-02-25T15:07:56.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f6449ba9640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:56.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f6449ba9640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:07:56.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f6449ba9640 10 monclient(hunting): picked mon.noname-c con 0x7f6444059e10 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:56.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f6449ba9640 10 monclient(hunting): picked mon.noname-a con 0x7f644415e2a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:56.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f6449ba9640 10 monclient(hunting): picked mon.noname-b con 0x7f64441a5660 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:56.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f6449ba9640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644412dd10 con 0x7f644415e2a0 2026-02-25T15:07:56.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f6449ba9640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644412dff0 con 0x7f64441a5660 2026-02-25T15:07:56.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f6449ba9640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644412e2d0 con 0x7f6444059e10 2026-02-25T15:07:56.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f6449ba9640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:56.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f6449ba9640 10 monclient(hunting): authenticate will time out at 3366.567159s 2026-02-25T15:07:56.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f6443fff640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f64441a5660 0x7f64441a4a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45652/0 (socket says 10.20.193.90:45652) 2026-02-25T15:07:56.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f6443fff640 1 -- 10.20.193.90:0/3456477225 learned_addr learned my addr 10.20.193.90:0/3456477225 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:56.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f641ffff640 10 client.?.objecter ms_handle_connect 0x7f64441a5660 2026-02-25T15:07:56.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f641ffff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:56.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f641ffff640 10 client.?.objecter ms_handle_connect 0x7f6444059e10 2026-02-25T15:07:56.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f641ffff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:56.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f641ffff640 10 client.?.objecter ms_handle_connect 0x7f644415e2a0 2026-02-25T15:07:56.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.967+0000 7f641ffff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:56.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.968+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3982994636 0 0) 0x7f644412dd10 con 0x7f644415e2a0 2026-02-25T15:07:56.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.968+0000 7f641ffff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.968+0000 7f641ffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.968+0000 7f641ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.968+0000 7f641ffff640 10 monclient(hunting): my global_id is 30858 2026-02-25T15:07:56.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.968+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6414003610 con 0x7f644415e2a0 2026-02-25T15:07:56.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.968+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1761361761 0 0) 0x7f644412dff0 con 0x7f64441a5660 2026-02-25T15:07:56.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.968+0000 7f641ffff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.968+0000 7f641ffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.968+0000 7f641ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.968+0000 7f641ffff640 10 monclient(hunting): my global_id is 27490 2026-02-25T15:07:56.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.968+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f644412dd10 con 0x7f64441a5660 2026-02-25T15:07:56.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.969+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 453249045 0 0) 0x7f644412e2d0 con 0x7f6444059e10 2026-02-25T15:07:56.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.969+0000 7f641ffff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:56.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.969+0000 7f641ffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:56.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.969+0000 7f641ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:56.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.969+0000 7f641ffff640 10 monclient(hunting): my global_id is 28628 2026-02-25T15:07:56.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.969+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f644412dff0 con 0x7f6444059e10 2026-02-25T15:07:56.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2860296359 0 0) 0x7f644412dd10 con 0x7f64441a5660 2026-02-25T15:07:56.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f644412e2d0 con 0x7f64441a5660 2026-02-25T15:07:56.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6438003490 con 0x7f64441a5660 2026-02-25T15:07:56.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:56.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:56.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 10 monclient(hunting): dump: 2026-02-25T15:07:56.982 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.983 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.983 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.983 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.983 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.983 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.983 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.983 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.983 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.983 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.983 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:56.983 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1470124019 0 0) 0x7f6414003610 con 0x7f644415e2a0 2026-02-25T15:07:56.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f644412dd10 con 0x7f644415e2a0 2026-02-25T15:07:56.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6434003310 con 0x7f644415e2a0 2026-02-25T15:07:56.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:56.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:56.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 10 monclient(hunting): dump: 2026-02-25T15:07:56.984 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.984 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.984 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.984 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.984 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.984 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.984 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.985 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.985 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.985 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.985 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2760619921 0 0) 0x7f644412dff0 con 0x7f6444059e10 2026-02-25T15:07:56.985 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6414003610 con 0x7f6444059e10 2026-02-25T15:07:56.985 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6430003060 con 0x7f6444059e10 2026-02-25T15:07:56.985 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:56.985 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:56.985 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.971+0000 7f641ffff640 10 monclient(hunting): dump: 2026-02-25T15:07:56.985 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.985 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.985 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.986 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.986 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.986 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.986 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.986 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.986 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.986 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.986 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.972+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1942900405 0 0) 0x7f644412e2d0 con 0x7f64441a5660 2026-02-25T15:07:56.986 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.972+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 >> v1:10.20.193.94:6789/0 conn(0x7f6444059e10 legacy=0x7f6444124480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.986 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.972+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 >> v1:10.20.193.90:6789/0 conn(0x7f644415e2a0 legacy=0x7f644412aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:56.986 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.972+0000 7f641ffff640 10 monclient: _finish_hunting 0 2026-02-25T15:07:56.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.972+0000 7f641ffff640 1 monclient: found mon.b 2026-02-25T15:07:56.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.972+0000 7f641ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:56.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.972+0000 7f641ffff640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:56.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.972+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644412e5b0 con 0x7f64441a5660 2026-02-25T15:07:56.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.972+0000 7f641ffff640 10 monclient: _finish_auth 0 2026-02-25T15:07:56.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f641ffff640 10 monclient: _check_auth_tickets 2026-02-25T15:07:56.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f641ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:56.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f6449ba9640 5 monclient: authenticate success, global_id 27490 2026-02-25T15:07:56.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f6449ba9640 10 monclient: _renew_subs 2026-02-25T15:07:56.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f6449ba9640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:56.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f6449ba9640 1 -- 10.20.193.90:0/3456477225 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f644412e890 con 0x7f64441a5660 2026-02-25T15:07:56.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f6449ba9640 10 client.27490.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:56.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f6449ba9640 10 monclient: _renew_subs 2026-02-25T15:07:56.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f6449ba9640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:56.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f6449ba9640 1 -- 10.20.193.90:0/3456477225 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f644412b320 con 0x7f64441a5660 2026-02-25T15:07:56.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f6449ba9640 1 librados: init done 2026-02-25T15:07:56.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f6449ba9640 10 client.27490.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:56.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f6449ba9640 15 client.27490.objecter _session_command_op_assign -1 1 2026-02-25T15:07:56.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f6449ba9640 20 client.27490.objecter _get_session osd=-1 returning homeless 2026-02-25T15:07:56.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f6449ba9640 10 client.27490.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:56.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f6449ba9640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:56.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.973+0000 7f6449ba9640 1 -- 10.20.193.90:0/3456477225 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f6444195e60 con 0x7f64441a5660 2026-02-25T15:07:56.990 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.981+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6444195e60 con 0x7f64441a5660 2026-02-25T15:07:56.990 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.981+0000 7f641ffff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:56.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.981+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6438003d30 con 0x7f64441a5660 2026-02-25T15:07:56.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.981+0000 7f641ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:56.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.981+0000 7f641ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:56.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.981+0000 7f6441ffb640 4 set_mon_vals no callback set 2026-02-25T15:07:56.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.981+0000 7f641ffff640 10 monclient: dump: 2026-02-25T15:07:56.991 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:56.991 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:56.991 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.991 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:56.991 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:56.991 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:56.992 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:56.992 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:56.992 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:56.992 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:56.992 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.982+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f6438003f40 con 0x7f64441a5660 2026-02-25T15:07:56.992 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f64437fe640 10 monclient: get_auth_request con 0x7f641407a0d0 auth_method 0 2026-02-25T15:07:56.992 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 10 client.27490.objecter ms_handle_connect 0x7f641407a0d0 2026-02-25T15:07:56.992 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7f6438003d30 con 0x7f64441a5660 2026-02-25T15:07:56.992 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 10 client.27490.objecter ms_dispatch2 0x7f6444000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:56.992 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 3 client.27490.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:56.992 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 3 client.27490.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:56.992 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 10 client.27490.objecter checking command 1 2026-02-25T15:07:56.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 20 client.27490.objecter _get_session s=0x7f64140a4380 osd=9 3 2026-02-25T15:07:56.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 20 client.27490.objecter put_session s=0x7f64140a4380 osd=9 3 2026-02-25T15:07:56.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 15 client.27490.objecter _session_command_op_remove -1 1 2026-02-25T15:07:56.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 20 client.27490.objecter _get_session s=0x7f64140a4380 osd=9 3 2026-02-25T15:07:56.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 20 client.27490.objecter get_session s=0x7f64140a4380 osd=9 3 2026-02-25T15:07:56.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 15 client.27490.objecter _session_command_op_assign 9 1 2026-02-25T15:07:56.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 20 client.27490.objecter put_session s=0x7f64140a4380 osd=9 4 2026-02-25T15:07:56.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 10 client.27490.objecter _send_command 1 2026-02-25T15:07:56.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 --> v1:10.20.193.93:6808/3039017972 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64140c6690 con 0x7f64140a4a30 2026-02-25T15:07:56.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.983+0000 7f641ffff640 20 client.27490.objecter dump_active .. 0 homeless 2026-02-25T15:07:56.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.984+0000 7f6442ffd640 10 monclient: get_auth_request con 0x7f64140a4a30 auth_method 0 2026-02-25T15:07:56.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.984+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_get_version_reply(handle=1 version=338) ==== 24+0+0 (unknown 1001891302 0 0) 0x7f643805b2b0 con 0x7f64441a5660 2026-02-25T15:07:56.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.984+0000 7f641ffff640 10 monclient: handle_get_version_reply finishing 1 version 338 2026-02-25T15:07:56.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.985+0000 7f641ffff640 10 client.27490.objecter ms_handle_connect 0x7f64140a4a30 2026-02-25T15:07:56.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.994+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== osd.9 v1:10.20.193.93:6808/3039017972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f64140c6690 con 0x7f64140a4a30 2026-02-25T15:07:56.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.994+0000 7f641ffff640 10 client.27490.objecter ms_dispatch2 0x7f6444000b90 command_reply(tid 1: 0 ) 2026-02-25T15:07:56.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.994+0000 7f641ffff640 10 client.27490.objecter _finish_command 1 = system:0 2026-02-25T15:07:56.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.994+0000 7f641ffff640 20 client.27490.objecter put_session s=0x7f64140a4380 osd=9 4 2026-02-25T15:07:56.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:56.994+0000 7f641ffff640 15 client.27490.objecter _session_command_op_remove 9 1 2026-02-25T15:07:57.003 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.002+0000 7f6449ba9640 10 client.27490.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:07:57.003 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.002+0000 7f6449ba9640 15 client.27490.objecter _session_command_op_assign -1 2 2026-02-25T15:07:57.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.002+0000 7f6449ba9640 20 client.27490.objecter _get_session s=0x7f64140a4380 osd=9 3 2026-02-25T15:07:57.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.002+0000 7f6449ba9640 20 client.27490.objecter put_session s=0x7f64140a4380 osd=9 3 2026-02-25T15:07:57.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.002+0000 7f6449ba9640 20 client.27490.objecter _get_session s=0x7f64140a4380 osd=9 3 2026-02-25T15:07:57.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.002+0000 7f6449ba9640 15 client.27490.objecter _session_command_op_remove -1 2 2026-02-25T15:07:57.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.002+0000 7f6449ba9640 20 client.27490.objecter get_session s=0x7f64140a4380 osd=9 3 2026-02-25T15:07:57.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.002+0000 7f6449ba9640 15 client.27490.objecter _session_command_op_assign 9 2 2026-02-25T15:07:57.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.002+0000 7f6449ba9640 20 client.27490.objecter put_session s=0x7f64140a4380 osd=9 4 2026-02-25T15:07:57.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.002+0000 7f6449ba9640 10 client.27490.objecter _send_command 2 2026-02-25T15:07:57.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.002+0000 7f6449ba9640 1 -- 10.20.193.90:0/3456477225 --> v1:10.20.193.93:6808/3039017972 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6444195b80 con 0x7f64140a4a30 2026-02-25T15:07:57.011 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.009+0000 7f641ffff640 1 -- 10.20.193.90:0/3456477225 <== osd.9 v1:10.20.193.93:6808/3039017972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f6444195b80 con 0x7f64140a4a30 2026-02-25T15:07:57.011 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.009+0000 7f641ffff640 10 client.27490.objecter ms_dispatch2 0x7f6444000b90 command_reply(tid 2: 0 ) 2026-02-25T15:07:57.011 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.009+0000 7f641ffff640 10 client.27490.objecter _finish_command 2 = system:0 2026-02-25T15:07:57.012 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.009+0000 7f641ffff640 20 client.27490.objecter put_session s=0x7f64140a4380 osd=9 4 2026-02-25T15:07:57.012 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.009+0000 7f641ffff640 15 client.27490.objecter _session_command_op_remove 9 2 2026-02-25T15:07:57.012 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:07:57.012 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-25T15:07:57.012 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:07:57.014 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 10 librados: watch_flush enter 2026-02-25T15:07:57.014 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 10 librados: watch_flush exit 2026-02-25T15:07:57.014 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 10 client.27490.objecter close_session for osd.9 2026-02-25T15:07:57.014 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 1 -- 10.20.193.90:0/3456477225 >> v1:10.20.193.93:6808/3039017972 conn(0x7f64140a4a30 legacy=0x7f64140c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.014 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 20 client.27490.objecter put_session s=0x7f64140a4380 osd=9 1 2026-02-25T15:07:57.014 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 20 client.27490.objecter shutdown clearing up homeless session... 2026-02-25T15:07:57.015 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 10 client.27490.objecter successfully canceled tick 2026-02-25T15:07:57.015 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 1 -- 10.20.193.90:0/3456477225 >> v1:10.20.193.189:6800/3647375982 conn(0x7f641407a0d0 legacy=0x7f641409a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.015 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 10 monclient: shutdown 2026-02-25T15:07:57.015 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:57.015 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 1 -- 10.20.193.90:0/3456477225 >> v1:10.20.193.93:6789/0 conn(0x7f64441a5660 legacy=0x7f64441a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.015 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6443fff640 1 -- 10.20.193.90:0/3456477225 reap_dead start 2026-02-25T15:07:57.015 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 1 -- 10.20.193.90:0/3456477225 shutdown_connections 2026-02-25T15:07:57.015 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 1 -- 10.20.193.90:0/3456477225 >> 10.20.193.90:0/3456477225 conn(0x7f64440b2f60 msgr2=0x7f64440b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:57.015 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 1 -- 10.20.193.90:0/3456477225 shutdown_connections 2026-02-25T15:07:57.015 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 1 -- 10.20.193.90:0/3456477225 wait complete. 2026-02-25T15:07:57.016 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.010+0000 7f6449ba9640 1 librados: shutdown 2026-02-25T15:07:57.021 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2026-02-25T15:07:57.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.089+0000 7f5954688640 10 monclient: get_monmap_and_config 2026-02-25T15:07:57.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.089+0000 7f5954688640 10 monclient: build_initial_monmap 2026-02-25T15:07:57.091 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.089+0000 7f5954688640 10 monclient: monmap: 2026-02-25T15:07:57.091 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:57.092 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.092 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:57.089852+0000 2026-02-25T15:07:57.092 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:57.089852+0000 2026-02-25T15:07:57.092 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:57.092 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.092 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:57.092 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:57.092 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:57.092 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.092 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.089+0000 7f5954688640 1 Processor -- start 2026-02-25T15:07:57.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.089+0000 7f5954688640 1 -- start start 2026-02-25T15:07:57.093 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.089+0000 7f5954688640 10 monclient: init 2026-02-25T15:07:57.099 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.090+0000 7f5954688640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:57.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.090+0000 7f5954688640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:07:57.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.090+0000 7f5954688640 10 monclient(hunting): picked mon.noname-a con 0x7f594c157780 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:57.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.090+0000 7f5954688640 10 monclient(hunting): picked mon.noname-b con 0x7f594c157b50 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:57.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.090+0000 7f5954688640 10 monclient(hunting): picked mon.noname-c con 0x7f594c164d00 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:57.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.090+0000 7f5954688640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f594c05de70 con 0x7f594c157780 2026-02-25T15:07:57.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.090+0000 7f5954688640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f594c055b20 con 0x7f594c157b50 2026-02-25T15:07:57.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.090+0000 7f5954688640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f594c055e00 con 0x7f594c164d00 2026-02-25T15:07:57.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.090+0000 7f5954688640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:57.100 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.090+0000 7f5954688640 10 monclient(hunting): authenticate will time out at 3366.690214s 2026-02-25T15:07:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.090+0000 7f5951bfc640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f594c157b50 0x7f594c1645f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45664/0 (socket says 10.20.193.90:45664) 2026-02-25T15:07:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.090+0000 7f5951bfc640 1 -- 10.20.193.90:0/190878386 learned_addr learned my addr 10.20.193.90:0/190878386 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.092+0000 7f59513fb640 1 -- 10.20.193.90:0/190878386 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1488837599 0 0) 0x7f594c055b20 con 0x7f594c157b50 2026-02-25T15:07:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.092+0000 7f59513fb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.092+0000 7f59513fb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.092+0000 7f59513fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.092+0000 7f59513fb640 10 monclient(hunting): my global_id is 27493 2026-02-25T15:07:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.092+0000 7f59513fb640 1 -- 10.20.193.90:0/190878386 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5934003610 con 0x7f594c157b50 2026-02-25T15:07:57.101 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.092+0000 7f59513fb640 1 -- 10.20.193.90:0/190878386 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2147396849 0 0) 0x7f594c055e00 con 0x7f594c164d00 2026-02-25T15:07:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.092+0000 7f59513fb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.092+0000 7f59513fb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.092+0000 7f59513fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.092+0000 7f59513fb640 10 monclient(hunting): my global_id is 28631 2026-02-25T15:07:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.092+0000 7f59513fb640 1 -- 10.20.193.90:0/190878386 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f594c055b20 con 0x7f594c164d00 2026-02-25T15:07:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 1 -- 10.20.193.90:0/190878386 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1435992289 0 0) 0x7f594c05de70 con 0x7f594c157780 2026-02-25T15:07:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.102 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 10 monclient(hunting): my global_id is 30861 2026-02-25T15:07:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 1 -- 10.20.193.90:0/190878386 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f594c055e00 con 0x7f594c157780 2026-02-25T15:07:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 1 -- 10.20.193.90:0/190878386 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3114212466 0 0) 0x7f5934003610 con 0x7f594c157b50 2026-02-25T15:07:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 1 -- 10.20.193.90:0/190878386 >> v1:10.20.193.94:6789/0 conn(0x7f594c164d00 legacy=0x7f594c1c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 1 -- 10.20.193.90:0/190878386 >> v1:10.20.193.90:6789/0 conn(0x7f594c157780 legacy=0x7f594c056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 10 monclient: _finish_hunting 0 2026-02-25T15:07:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 1 monclient: found mon.noname-b 2026-02-25T15:07:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:07:57.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 1 -- 10.20.193.90:0/190878386 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594c0560e0 con 0x7f594c157b50 2026-02-25T15:07:57.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 10 monclient: _finish_auth 0 2026-02-25T15:07:57.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 10 monclient: _check_auth_tickets 2026-02-25T15:07:57.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:57.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 1 -- 10.20.193.90:0/190878386 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5940002df0 con 0x7f594c157b50 2026-02-25T15:07:57.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:57.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:57.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f59513fb640 10 monclient: dump: 2026-02-25T15:07:57.104 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.104 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.104 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.105 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.105 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.105 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.105 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.105 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.105 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.105 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f5954688640 5 monclient: authenticate success, global_id 27493 2026-02-25T15:07:57.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.093+0000 7f5954688640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:57.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.094+0000 7f59513fb640 1 -- 10.20.193.90:0/190878386 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5940002fc0 con 0x7f594c157b50 2026-02-25T15:07:57.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.094+0000 7f59513fb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:57.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.094+0000 7f59513fb640 1 -- 10.20.193.90:0/190878386 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5940003400 con 0x7f594c157b50 2026-02-25T15:07:57.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.094+0000 7f59513fb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:57.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.094+0000 7f59513fb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:57.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.094+0000 7f59513fb640 10 monclient: dump: 2026-02-25T15:07:57.106 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.106 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.106 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.106 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.107 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.107 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.107 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.107 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.107 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.107 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.094+0000 7f5954688640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:57.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.094+0000 7f5954688640 4 set_mon_vals no callback set 2026-02-25T15:07:57.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 10 monclient: shutdown 2026-02-25T15:07:57.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:57.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 1 -- 10.20.193.90:0/190878386 >> v1:10.20.193.93:6789/0 conn(0x7f594c157b50 legacy=0x7f594c1645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 1 -- 10.20.193.90:0/190878386 shutdown_connections 2026-02-25T15:07:57.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 1 -- 10.20.193.90:0/190878386 >> 10.20.193.90:0/190878386 conn(0x7f594c0b2ef0 msgr2=0x7f594c0b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:57.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 1 -- 10.20.193.90:0/190878386 shutdown_connections 2026-02-25T15:07:57.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 1 -- 10.20.193.90:0/190878386 wait complete. 2026-02-25T15:07:57.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 10 monclient: build_initial_monmap 2026-02-25T15:07:57.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 10 monclient: monmap: 2026-02-25T15:07:57.108 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:57.108 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:57.108 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:57.108 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:57.109 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:57.109 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:57.109 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:57.109 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:57.109 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:57.109 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 1 librados: starting msgr at 2026-02-25T15:07:57.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 1 librados: starting objecter 2026-02-25T15:07:57.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:57.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 1 Processor -- start 2026-02-25T15:07:57.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 1 -- start start 2026-02-25T15:07:57.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 1 librados: setting wanted keys 2026-02-25T15:07:57.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 1 librados: calling monclient init 2026-02-25T15:07:57.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.095+0000 7f5954688640 10 monclient: init 2026-02-25T15:07:57.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5954688640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:57.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5954688640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:07:57.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5954688640 10 monclient(hunting): picked mon.noname-b con 0x7f594c157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:57.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5954688640 10 monclient(hunting): picked mon.noname-c con 0x7f594c157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:57.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5954688640 10 monclient(hunting): picked mon.noname-a con 0x7f594c164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:57.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5954688640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f594c0af1e0 con 0x7f594c164d00 2026-02-25T15:07:57.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5954688640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f594c0af4c0 con 0x7f594c157780 2026-02-25T15:07:57.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5954688640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f594c0af7a0 con 0x7f594c157b50 2026-02-25T15:07:57.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5954688640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:57.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5954688640 10 monclient(hunting): authenticate will time out at 3366.696141s 2026-02-25T15:07:57.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5952bfe640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f594c164d00 0x7f594c05c750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39840/0 (socket says 10.20.193.90:39840) 2026-02-25T15:07:57.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5952bfe640 1 -- 10.20.193.90:0/1664564856 learned_addr learned my addr 10.20.193.90:0/1664564856 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:57.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5932ffd640 10 client.?.objecter ms_handle_connect 0x7f594c164d00 2026-02-25T15:07:57.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5932ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:57.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5932ffd640 10 client.?.objecter ms_handle_connect 0x7f594c157780 2026-02-25T15:07:57.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5932ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:57.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5932ffd640 10 client.?.objecter ms_handle_connect 0x7f594c157b50 2026-02-25T15:07:57.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.096+0000 7f5932ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:57.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 850930431 0 0) 0x7f594c0af1e0 con 0x7f594c164d00 2026-02-25T15:07:57.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 10 monclient(hunting): my global_id is 30864 2026-02-25T15:07:57.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f591c003610 con 0x7f594c164d00 2026-02-25T15:07:57.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3350000390 0 0) 0x7f594c0af4c0 con 0x7f594c157780 2026-02-25T15:07:57.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 10 monclient(hunting): my global_id is 27496 2026-02-25T15:07:57.113 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f594c0af1e0 con 0x7f594c157780 2026-02-25T15:07:57.113 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 555183058 0 0) 0x7f594c0af7a0 con 0x7f594c157b50 2026-02-25T15:07:57.113 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.113 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.113 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.113 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 10 monclient(hunting): my global_id is 28634 2026-02-25T15:07:57.113 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.097+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f594c0af4c0 con 0x7f594c157b50 2026-02-25T15:07:57.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.099+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3614873530 0 0) 0x7f591c003610 con 0x7f594c164d00 2026-02-25T15:07:57.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.099+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f594c0af7a0 con 0x7f594c164d00 2026-02-25T15:07:57.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.099+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5948003360 con 0x7f594c164d00 2026-02-25T15:07:57.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.099+0000 7f5932ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:57.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.099+0000 7f5932ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:57.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.099+0000 7f5932ffd640 10 monclient(hunting): dump: 2026-02-25T15:07:57.117 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.117 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.117 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.117 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.117 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.117 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.117 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.117 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.117 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.117 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.099+0000 7f5932ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:57.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.100+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3616084647 0 0) 0x7f594c0af1e0 con 0x7f594c157780 2026-02-25T15:07:57.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.100+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f591c003610 con 0x7f594c157780 2026-02-25T15:07:57.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.100+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f593c0030d0 con 0x7f594c157780 2026-02-25T15:07:57.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.100+0000 7f5932ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:57.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.100+0000 7f5932ffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:57.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.100+0000 7f5932ffd640 10 monclient(hunting): dump: 2026-02-25T15:07:57.118 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.118 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.118 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.118 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.119 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.119 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.119 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.119 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.119 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.119 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.100+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3455336184 0 0) 0x7f594c0af4c0 con 0x7f594c157b50 2026-02-25T15:07:57.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.100+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f594c0af1e0 con 0x7f594c157b50 2026-02-25T15:07:57.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.100+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5940003310 con 0x7f594c157b50 2026-02-25T15:07:57.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.100+0000 7f5932ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:57.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.100+0000 7f5932ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:57.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.100+0000 7f5932ffd640 10 monclient(hunting): dump: 2026-02-25T15:07:57.120 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.120 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.120 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.120 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.120 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.120 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.120 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.120 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.120 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.120 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2645288476 0 0) 0x7f594c0af7a0 con 0x7f594c164d00 2026-02-25T15:07:57.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 >> v1:10.20.193.94:6789/0 conn(0x7f594c157b50 legacy=0x7f594c14dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 >> v1:10.20.193.93:6789/0 conn(0x7f594c157780 legacy=0x7f594c14ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5932ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:07:57.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5932ffd640 1 monclient: found mon.a 2026-02-25T15:07:57.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5932ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:57.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5932ffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594c0afa80 con 0x7f594c164d00 2026-02-25T15:07:57.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5932ffd640 10 monclient: _finish_auth 0 2026-02-25T15:07:57.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5932ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:07:57.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5932ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:57.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5954688640 5 monclient: authenticate success, global_id 30864 2026-02-25T15:07:57.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5954688640 10 monclient: _renew_subs 2026-02-25T15:07:57.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5954688640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5954688640 1 -- 10.20.193.90:0/1664564856 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f594c0afd60 con 0x7f594c164d00 2026-02-25T15:07:57.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5954688640 10 client.30864.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:57.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5954688640 10 monclient: _renew_subs 2026-02-25T15:07:57.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5954688640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5954688640 1 -- 10.20.193.90:0/1664564856 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f594c14c190 con 0x7f594c164d00 2026-02-25T15:07:57.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.101+0000 7f5954688640 1 librados: init done 2026-02-25T15:07:57.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.102+0000 7f5954688640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:57.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.102+0000 7f5954688640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:57.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.102+0000 7f5954688640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.102+0000 7f5954688640 1 -- 10.20.193.90:0/1664564856 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f594c0b5870 con 0x7f594c164d00 2026-02-25T15:07:57.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.191+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5948003a80 con 0x7f594c164d00 2026-02-25T15:07:57.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.191+0000 7f5932ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:57.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.191+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f5948003e50 con 0x7f594c164d00 2026-02-25T15:07:57.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.191+0000 7f5932ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:57.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.191+0000 7f5932ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:57.193 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.191+0000 7f5932ffd640 10 monclient: dump: 2026-02-25T15:07:57.193 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.193 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.193 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.194 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.194 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.194 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.194 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.194 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.194 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.194 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.191+0000 7f5950bfa640 4 set_mon_vals no callback set 2026-02-25T15:07:57.194 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.191+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f5948004030 con 0x7f594c164d00 2026-02-25T15:07:57.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.192+0000 7f59523fd640 10 monclient: get_auth_request con 0x7f591c07a090 auth_method 0 2026-02-25T15:07:57.195 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.192+0000 7f5932ffd640 10 client.30864.objecter ms_handle_connect 0x7f591c07a090 2026-02-25T15:07:57.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.194+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7f5948095d60 con 0x7f594c164d00 2026-02-25T15:07:57.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.194+0000 7f5932ffd640 10 client.30864.objecter ms_dispatch2 0x7f594c000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:57.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.194+0000 7f5932ffd640 3 client.30864.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:57.196 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.194+0000 7f5932ffd640 3 client.30864.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:57.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.194+0000 7f5932ffd640 20 client.30864.objecter dump_active .. 0 homeless 2026-02-25T15:07:57.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.194+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f5948003e50 con 0x7f594c164d00 2026-02-25T15:07:57.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.194+0000 7f5932ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:57.197 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.194+0000 7f5932ffd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:57.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.304+0000 7f5954688640 1 -- 10.20.193.90:0/1664564856 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "9", "target": ["mon-mgr", ""]}) -- 0x7f594c0b5590 con 0x7f591c07a090 2026-02-25T15:07:57.306 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.304+0000 7f5932ffd640 1 -- 10.20.193.90:0/1664564856 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 9 to deep-scrub) ==== 41+0+0 (unknown 795720238 0 0) 0x7f594c0b5590 con 0x7f591c07a090 2026-02-25T15:07:57.306 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 9 to deep-scrub 2026-02-25T15:07:57.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.306+0000 7f5954688640 10 librados: watch_flush enter 2026-02-25T15:07:57.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.306+0000 7f5954688640 10 librados: watch_flush exit 2026-02-25T15:07:57.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.306+0000 7f5954688640 20 client.30864.objecter shutdown clearing up homeless session... 2026-02-25T15:07:57.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.306+0000 7f5954688640 10 client.30864.objecter successfully canceled tick 2026-02-25T15:07:57.308 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.306+0000 7f5954688640 1 -- 10.20.193.90:0/1664564856 >> v1:10.20.193.189:6800/3647375982 conn(0x7f591c07a090 legacy=0x7f591c09a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.306+0000 7f5954688640 10 monclient: shutdown 2026-02-25T15:07:57.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.306+0000 7f5954688640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:57.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.306+0000 7f5954688640 1 -- 10.20.193.90:0/1664564856 >> v1:10.20.193.90:6789/0 conn(0x7f594c164d00 legacy=0x7f594c05c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.306+0000 7f5954688640 1 -- 10.20.193.90:0/1664564856 shutdown_connections 2026-02-25T15:07:57.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.306+0000 7f5954688640 1 -- 10.20.193.90:0/1664564856 >> 10.20.193.90:0/1664564856 conn(0x7f594c0b2ef0 msgr2=0x7f594c0ad9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:57.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.306+0000 7f5954688640 1 -- 10.20.193.90:0/1664564856 shutdown_connections 2026-02-25T15:07:57.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.306+0000 7f5954688640 1 -- 10.20.193.90:0/1664564856 wait complete. 2026-02-25T15:07:57.309 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.306+0000 7f5954688640 1 librados: shutdown 2026-02-25T15:07:57.317 INFO:tasks.ceph:Scrubbing osd.13 2026-02-25T15:07:57.317 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:07:57.389 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.387+0000 7fe1d2734640 10 monclient: get_monmap_and_config 2026-02-25T15:07:57.389 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.387+0000 7fe1d2734640 10 monclient: build_initial_monmap 2026-02-25T15:07:57.389 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.387+0000 7fe1d2734640 10 monclient: monmap: 2026-02-25T15:07:57.389 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:57.389 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.389 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:57.387763+0000 2026-02-25T15:07:57.390 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:57.387763+0000 2026-02-25T15:07:57.390 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:57.390 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.390 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:57.390 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:57.390 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:57.390 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.387+0000 7fe1d2734640 1 Processor -- start 2026-02-25T15:07:57.390 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.387+0000 7fe1d2734640 1 -- start start 2026-02-25T15:07:57.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.387+0000 7fe1d2734640 10 monclient: init 2026-02-25T15:07:57.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.388+0000 7fe1d2734640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:57.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.388+0000 7fe1d2734640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:07:57.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.388+0000 7fe1d2734640 10 monclient(hunting): picked mon.noname-c con 0x7fe1cc15e2a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:57.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.388+0000 7fe1d2734640 10 monclient(hunting): picked mon.noname-a con 0x7fe1cc059e10 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:57.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.388+0000 7fe1d2734640 10 monclient(hunting): picked mon.noname-b con 0x7fe1cc1a5660 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:57.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.388+0000 7fe1d2734640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1cc162df0 con 0x7fe1cc059e10 2026-02-25T15:07:57.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.388+0000 7fe1d2734640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1cc1630d0 con 0x7fe1cc1a5660 2026-02-25T15:07:57.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.388+0000 7fe1d2734640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1cc1633b0 con 0x7fe1cc15e2a0 2026-02-25T15:07:57.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.388+0000 7fe1d2734640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:57.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.388+0000 7fe1d2734640 10 monclient(hunting): authenticate will time out at 3366.988229s 2026-02-25T15:07:57.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.388+0000 7fe1d0caa640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fe1cc1a5660 0x7fe1cc1c5a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45682/0 (socket says 10.20.193.90:45682) 2026-02-25T15:07:57.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.388+0000 7fe1d0caa640 1 -- 10.20.193.90:0/3086202553 learned_addr learned my addr 10.20.193.90:0/3086202553 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:57.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.389+0000 7fe1caffd640 1 -- 10.20.193.90:0/3086202553 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2541392863 0 0) 0x7fe1cc1633b0 con 0x7fe1cc15e2a0 2026-02-25T15:07:57.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.389+0000 7fe1caffd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.389+0000 7fe1caffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.389+0000 7fe1caffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.389+0000 7fe1caffd640 10 monclient(hunting): my global_id is 28637 2026-02-25T15:07:57.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.389+0000 7fe1caffd640 1 -- 10.20.193.90:0/3086202553 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1b0003610 con 0x7fe1cc15e2a0 2026-02-25T15:07:57.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.389+0000 7fe1caffd640 1 -- 10.20.193.90:0/3086202553 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1300152979 0 0) 0x7fe1cc1630d0 con 0x7fe1cc1a5660 2026-02-25T15:07:57.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.389+0000 7fe1caffd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.389+0000 7fe1caffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.389+0000 7fe1caffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.389+0000 7fe1caffd640 10 monclient(hunting): my global_id is 27499 2026-02-25T15:07:57.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.389+0000 7fe1caffd640 1 -- 10.20.193.90:0/3086202553 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1cc1633b0 con 0x7fe1cc1a5660 2026-02-25T15:07:57.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.390+0000 7fe1caffd640 1 -- 10.20.193.90:0/3086202553 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 925447780 0 0) 0x7fe1cc162df0 con 0x7fe1cc059e10 2026-02-25T15:07:57.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.390+0000 7fe1caffd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.390+0000 7fe1caffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.390+0000 7fe1caffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.390+0000 7fe1caffd640 10 monclient(hunting): my global_id is 30867 2026-02-25T15:07:57.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.390+0000 7fe1caffd640 1 -- 10.20.193.90:0/3086202553 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1cc1630d0 con 0x7fe1cc059e10 2026-02-25T15:07:57.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1caffd640 1 -- 10.20.193.90:0/3086202553 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1497781430 0 0) 0x7fe1b0003610 con 0x7fe1cc15e2a0 2026-02-25T15:07:57.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1caffd640 1 -- 10.20.193.90:0/3086202553 >> v1:10.20.193.93:6789/0 conn(0x7fe1cc1a5660 legacy=0x7fe1cc1c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1caffd640 1 -- 10.20.193.90:0/3086202553 >> v1:10.20.193.90:6789/0 conn(0x7fe1cc059e10 legacy=0x7fe1cc15ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1caffd640 10 monclient: _finish_hunting 0 2026-02-25T15:07:57.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1caffd640 1 monclient: found mon.noname-c 2026-02-25T15:07:57.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1caffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:57.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1caffd640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:07:57.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1caffd640 1 -- 10.20.193.90:0/3086202553 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc163690 con 0x7fe1cc15e2a0 2026-02-25T15:07:57.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1caffd640 10 monclient: _finish_auth 0 2026-02-25T15:07:57.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1caffd640 10 monclient: _check_auth_tickets 2026-02-25T15:07:57.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1caffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:57.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1caffd640 1 -- 10.20.193.90:0/3086202553 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe1b8002df0 con 0x7fe1cc15e2a0 2026-02-25T15:07:57.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1caffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:57.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1caffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:07:57.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1caffd640 10 monclient: dump: 2026-02-25T15:07:57.401 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.402 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.402 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.402 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.402 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.402 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.402 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.402 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.402 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.402 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1d2734640 5 monclient: authenticate success, global_id 28637 2026-02-25T15:07:57.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.391+0000 7fe1d2734640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:57.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.392+0000 7fe1caffd640 1 -- 10.20.193.90:0/3086202553 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe1b8002fc0 con 0x7fe1cc15e2a0 2026-02-25T15:07:57.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.392+0000 7fe1caffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:57.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.392+0000 7fe1caffd640 1 -- 10.20.193.90:0/3086202553 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe1b8003400 con 0x7fe1cc15e2a0 2026-02-25T15:07:57.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.392+0000 7fe1caffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:57.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.392+0000 7fe1caffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:57.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.392+0000 7fe1caffd640 10 monclient: dump: 2026-02-25T15:07:57.403 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.403 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.403 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.403 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.404 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.404 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.404 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.404 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.404 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.404 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.392+0000 7fe1d2734640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:57.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.392+0000 7fe1d2734640 4 set_mon_vals no callback set 2026-02-25T15:07:57.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.392+0000 7fe1d2734640 10 monclient: shutdown 2026-02-25T15:07:57.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.392+0000 7fe1d2734640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:57.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.392+0000 7fe1d2734640 1 -- 10.20.193.90:0/3086202553 >> v1:10.20.193.94:6789/0 conn(0x7fe1cc15e2a0 legacy=0x7fe1cc05da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.392+0000 7fe1d2734640 1 -- 10.20.193.90:0/3086202553 shutdown_connections 2026-02-25T15:07:57.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.392+0000 7fe1d2734640 1 -- 10.20.193.90:0/3086202553 >> 10.20.193.90:0/3086202553 conn(0x7fe1cc0b2f60 msgr2=0x7fe1cc0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:57.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.392+0000 7fe1d2734640 1 -- 10.20.193.90:0/3086202553 shutdown_connections 2026-02-25T15:07:57.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.393+0000 7fe1d2734640 1 -- 10.20.193.90:0/3086202553 wait complete. 2026-02-25T15:07:57.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.393+0000 7fe1d2734640 10 monclient: build_initial_monmap 2026-02-25T15:07:57.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.393+0000 7fe1d2734640 10 monclient: monmap: 2026-02-25T15:07:57.405 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:57.405 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:57.405 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:57.406 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:57.406 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:57.406 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:57.406 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:57.406 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:57.406 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:57.406 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.393+0000 7fe1d2734640 1 librados: starting msgr at 2026-02-25T15:07:57.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.393+0000 7fe1d2734640 1 librados: starting objecter 2026-02-25T15:07:57.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.393+0000 7fe1d2734640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:57.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.393+0000 7fe1d2734640 1 Processor -- start 2026-02-25T15:07:57.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.393+0000 7fe1d2734640 1 -- start start 2026-02-25T15:07:57.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.393+0000 7fe1d2734640 1 librados: setting wanted keys 2026-02-25T15:07:57.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.393+0000 7fe1d2734640 1 librados: calling monclient init 2026-02-25T15:07:57.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.393+0000 7fe1d2734640 10 monclient: init 2026-02-25T15:07:57.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.394+0000 7fe1d2734640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:57.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.394+0000 7fe1d2734640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:07:57.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.394+0000 7fe1d2734640 10 monclient(hunting): picked mon.noname-c con 0x7fe1cc059e10 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:57.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.394+0000 7fe1d2734640 10 monclient(hunting): picked mon.noname-b con 0x7fe1cc15e2a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:57.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.394+0000 7fe1d2734640 10 monclient(hunting): picked mon.noname-a con 0x7fe1cc1a5660 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:57.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.394+0000 7fe1d2734640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1cc193b10 con 0x7fe1cc1a5660 2026-02-25T15:07:57.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.394+0000 7fe1d2734640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1cc056d10 con 0x7fe1cc15e2a0 2026-02-25T15:07:57.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.394+0000 7fe1d2734640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1cc056ff0 con 0x7fe1cc059e10 2026-02-25T15:07:57.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.394+0000 7fe1d2734640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:57.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.394+0000 7fe1d2734640 10 monclient(hunting): authenticate will time out at 3366.994205s 2026-02-25T15:07:57.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.394+0000 7fe1d0caa640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fe1cc1a5660 0x7fe1cc123c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39862/0 (socket says 10.20.193.90:39862) 2026-02-25T15:07:57.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.394+0000 7fe1d0caa640 1 -- 10.20.193.90:0/4096300294 learned_addr learned my addr 10.20.193.90:0/4096300294 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:57.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.394+0000 7fe1c8ff9640 10 client.?.objecter ms_handle_connect 0x7fe1cc1a5660 2026-02-25T15:07:57.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.394+0000 7fe1c8ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:57.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 10 client.?.objecter ms_handle_connect 0x7fe1cc059e10 2026-02-25T15:07:57.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:57.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1527639486 0 0) 0x7fe1cc193b10 con 0x7fe1cc1a5660 2026-02-25T15:07:57.408 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 10 monclient(hunting): my global_id is 30870 2026-02-25T15:07:57.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe19c003610 con 0x7fe1cc1a5660 2026-02-25T15:07:57.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 10 client.?.objecter ms_handle_connect 0x7fe1cc15e2a0 2026-02-25T15:07:57.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:57.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 9081207 0 0) 0x7fe1cc056ff0 con 0x7fe1cc059e10 2026-02-25T15:07:57.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.409 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 10 monclient(hunting): my global_id is 28640 2026-02-25T15:07:57.410 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.395+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1cc193b10 con 0x7fe1cc059e10 2026-02-25T15:07:57.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.396+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1049649333 0 0) 0x7fe1cc056d10 con 0x7fe1cc15e2a0 2026-02-25T15:07:57.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.396+0000 7fe1c8ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.396+0000 7fe1c8ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.396+0000 7fe1c8ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.396+0000 7fe1c8ff9640 10 monclient(hunting): my global_id is 27502 2026-02-25T15:07:57.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.396+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1cc056ff0 con 0x7fe1cc15e2a0 2026-02-25T15:07:57.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.397+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 487932996 0 0) 0x7fe19c003610 con 0x7fe1cc1a5660 2026-02-25T15:07:57.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.397+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe1cc056d10 con 0x7fe1cc1a5660 2026-02-25T15:07:57.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.397+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe1c0003430 con 0x7fe1cc1a5660 2026-02-25T15:07:57.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.397+0000 7fe1c8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:57.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.397+0000 7fe1c8ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:57.415 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.397+0000 7fe1c8ff9640 10 monclient(hunting): dump: 2026-02-25T15:07:57.415 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.415 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.415 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.416 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.416 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.416 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.416 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.416 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.416 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.416 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.397+0000 7fe1c8ff9640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:57.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.398+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3665128261 0 0) 0x7fe1cc193b10 con 0x7fe1cc059e10 2026-02-25T15:07:57.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.398+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe19c003610 con 0x7fe1cc059e10 2026-02-25T15:07:57.416 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.398+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe1b8003310 con 0x7fe1cc059e10 2026-02-25T15:07:57.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.398+0000 7fe1c8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:57.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.398+0000 7fe1c8ff9640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:57.417 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.398+0000 7fe1c8ff9640 10 monclient(hunting): dump: 2026-02-25T15:07:57.417 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.417 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.417 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.417 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.417 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.417 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.417 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.417 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.418 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.418 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4144434378 0 0) 0x7fe1cc056ff0 con 0x7fe1cc15e2a0 2026-02-25T15:07:57.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe1cc193b10 con 0x7fe1cc15e2a0 2026-02-25T15:07:57.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe1bc002fb0 con 0x7fe1cc15e2a0 2026-02-25T15:07:57.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:57.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:57.418 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 10 monclient(hunting): dump: 2026-02-25T15:07:57.418 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.418 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.418 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.419 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.419 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.419 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.419 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.419 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.419 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.419 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3148336520 0 0) 0x7fe1cc056d10 con 0x7fe1cc1a5660 2026-02-25T15:07:57.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 >> v1:10.20.193.94:6789/0 conn(0x7fe1cc059e10 legacy=0x7fe1cc193400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 >> v1:10.20.193.93:6789/0 conn(0x7fe1cc15e2a0 legacy=0x7fe1cc0b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 10 monclient: _finish_hunting 0 2026-02-25T15:07:57.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 1 monclient: found mon.a 2026-02-25T15:07:57.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:57.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc0572d0 con 0x7fe1cc1a5660 2026-02-25T15:07:57.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 10 monclient: _finish_auth 0 2026-02-25T15:07:57.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 10 monclient: _check_auth_tickets 2026-02-25T15:07:57.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1c8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:57.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1d2734640 5 monclient: authenticate success, global_id 30870 2026-02-25T15:07:57.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1d2734640 10 monclient: _renew_subs 2026-02-25T15:07:57.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1d2734640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1d2734640 1 -- 10.20.193.90:0/4096300294 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe1cc0575b0 con 0x7fe1cc1a5660 2026-02-25T15:07:57.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1d2734640 10 client.30870.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:57.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1d2734640 10 monclient: _renew_subs 2026-02-25T15:07:57.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1d2734640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1d2734640 1 -- 10.20.193.90:0/4096300294 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe1cc057b70 con 0x7fe1cc1a5660 2026-02-25T15:07:57.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.399+0000 7fe1d2734640 1 librados: init done 2026-02-25T15:07:57.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.400+0000 7fe1d2734640 10 client.30870.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:57.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.400+0000 7fe1d2734640 15 client.30870.objecter _session_command_op_assign -1 1 2026-02-25T15:07:57.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.400+0000 7fe1d2734640 20 client.30870.objecter _get_session osd=-1 returning homeless 2026-02-25T15:07:57.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.400+0000 7fe1d2734640 10 client.30870.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:57.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.400+0000 7fe1d2734640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.400+0000 7fe1d2734640 1 -- 10.20.193.90:0/4096300294 --> v1:10.20.193.90:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fe1cc194e50 con 0x7fe1cc1a5660 2026-02-25T15:07:57.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.408+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe1cc194e50 con 0x7fe1cc1a5660 2026-02-25T15:07:57.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.408+0000 7fe1c8ff9640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:57.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.408+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe1c0004df0 con 0x7fe1cc1a5660 2026-02-25T15:07:57.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.408+0000 7fe1c8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:57.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.408+0000 7fe1c8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:57.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.408+0000 7fe1ca7fc640 4 set_mon_vals no callback set 2026-02-25T15:07:57.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.408+0000 7fe1c8ff9640 10 monclient: dump: 2026-02-25T15:07:57.422 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.422 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.423 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.423 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.423 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.423 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.423 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.423 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.423 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.423 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.409+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fe1c001e2d0 con 0x7fe1cc1a5660 2026-02-25T15:07:57.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.409+0000 7fe1cbfff640 10 monclient: get_auth_request con 0x7fe19c07a120 auth_method 0 2026-02-25T15:07:57.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 10 client.30870.objecter ms_handle_connect 0x7fe19c07a120 2026-02-25T15:07:57.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7fe1c0095c70 con 0x7fe1cc1a5660 2026-02-25T15:07:57.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 10 client.30870.objecter ms_dispatch2 0x7fe1cc000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:57.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 3 client.30870.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:57.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 3 client.30870.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:57.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 10 client.30870.objecter checking command 1 2026-02-25T15:07:57.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 20 client.30870.objecter _get_session s=0x7fe19c0a43d0 osd=13 3 2026-02-25T15:07:57.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 20 client.30870.objecter put_session s=0x7fe19c0a43d0 osd=13 3 2026-02-25T15:07:57.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 15 client.30870.objecter _session_command_op_remove -1 1 2026-02-25T15:07:57.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 20 client.30870.objecter _get_session s=0x7fe19c0a43d0 osd=13 3 2026-02-25T15:07:57.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 20 client.30870.objecter get_session s=0x7fe19c0a43d0 osd=13 3 2026-02-25T15:07:57.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 15 client.30870.objecter _session_command_op_assign 13 1 2026-02-25T15:07:57.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 20 client.30870.objecter put_session s=0x7fe19c0a43d0 osd=13 4 2026-02-25T15:07:57.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 10 client.30870.objecter _send_command 1 2026-02-25T15:07:57.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 --> v1:10.20.193.93:6812/1850919728 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe19c0c66e0 con 0x7fe19c0a4a80 2026-02-25T15:07:57.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 20 client.30870.objecter dump_active .. 0 homeless 2026-02-25T15:07:57.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_get_version_reply(handle=1 version=338) ==== 24+0+0 (unknown 1001891302 0 0) 0x7fe1c005b240 con 0x7fe1cc1a5660 2026-02-25T15:07:57.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.410+0000 7fe1c8ff9640 10 monclient: handle_get_version_reply finishing 1 version 338 2026-02-25T15:07:57.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.411+0000 7fe1cb7fe640 10 monclient: get_auth_request con 0x7fe19c0a4a80 auth_method 0 2026-02-25T15:07:57.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.412+0000 7fe1c8ff9640 10 client.30870.objecter ms_handle_connect 0x7fe19c0a4a80 2026-02-25T15:07:57.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.424+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== osd.13 v1:10.20.193.93:6812/1850919728 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7fe19c0c66e0 con 0x7fe19c0a4a80 2026-02-25T15:07:57.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.424+0000 7fe1c8ff9640 10 client.30870.objecter ms_dispatch2 0x7fe1cc000b90 command_reply(tid 1: 0 ) 2026-02-25T15:07:57.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.424+0000 7fe1c8ff9640 10 client.30870.objecter _finish_command 1 = system:0 2026-02-25T15:07:57.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.424+0000 7fe1c8ff9640 20 client.30870.objecter put_session s=0x7fe19c0a43d0 osd=13 4 2026-02-25T15:07:57.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.424+0000 7fe1c8ff9640 15 client.30870.objecter _session_command_op_remove 13 1 2026-02-25T15:07:57.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.431+0000 7fe1d2734640 10 client.30870.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:07:57.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.431+0000 7fe1d2734640 15 client.30870.objecter _session_command_op_assign -1 2 2026-02-25T15:07:57.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.431+0000 7fe1d2734640 20 client.30870.objecter _get_session s=0x7fe19c0a43d0 osd=13 3 2026-02-25T15:07:57.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.431+0000 7fe1d2734640 20 client.30870.objecter put_session s=0x7fe19c0a43d0 osd=13 3 2026-02-25T15:07:57.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.431+0000 7fe1d2734640 20 client.30870.objecter _get_session s=0x7fe19c0a43d0 osd=13 3 2026-02-25T15:07:57.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.431+0000 7fe1d2734640 15 client.30870.objecter _session_command_op_remove -1 2 2026-02-25T15:07:57.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.431+0000 7fe1d2734640 20 client.30870.objecter get_session s=0x7fe19c0a43d0 osd=13 3 2026-02-25T15:07:57.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.431+0000 7fe1d2734640 15 client.30870.objecter _session_command_op_assign 13 2 2026-02-25T15:07:57.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.431+0000 7fe1d2734640 20 client.30870.objecter put_session s=0x7fe19c0a43d0 osd=13 4 2026-02-25T15:07:57.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.431+0000 7fe1d2734640 10 client.30870.objecter _send_command 2 2026-02-25T15:07:57.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.431+0000 7fe1d2734640 1 -- 10.20.193.90:0/4096300294 --> v1:10.20.193.93:6812/1850919728 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe1cc194b70 con 0x7fe19c0a4a80 2026-02-25T15:07:57.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.439+0000 7fe1c8ff9640 1 -- 10.20.193.90:0/4096300294 <== osd.13 v1:10.20.193.93:6812/1850919728 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7fe1cc194b70 con 0x7fe19c0a4a80 2026-02-25T15:07:57.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.439+0000 7fe1c8ff9640 10 client.30870.objecter ms_dispatch2 0x7fe1cc000b90 command_reply(tid 2: 0 ) 2026-02-25T15:07:57.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.439+0000 7fe1c8ff9640 10 client.30870.objecter _finish_command 2 = system:0 2026-02-25T15:07:57.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.439+0000 7fe1c8ff9640 20 client.30870.objecter put_session s=0x7fe19c0a43d0 osd=13 4 2026-02-25T15:07:57.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.439+0000 7fe1c8ff9640 15 client.30870.objecter _session_command_op_remove 13 2 2026-02-25T15:07:57.442 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:07:57.442 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-25T15:07:57.442 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:07:57.443 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 10 librados: watch_flush enter 2026-02-25T15:07:57.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 10 librados: watch_flush exit 2026-02-25T15:07:57.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 10 client.30870.objecter close_session for osd.13 2026-02-25T15:07:57.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 1 -- 10.20.193.90:0/4096300294 >> v1:10.20.193.93:6812/1850919728 conn(0x7fe19c0a4a80 legacy=0x7fe19c0c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 20 client.30870.objecter put_session s=0x7fe19c0a43d0 osd=13 1 2026-02-25T15:07:57.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 20 client.30870.objecter shutdown clearing up homeless session... 2026-02-25T15:07:57.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 10 client.30870.objecter successfully canceled tick 2026-02-25T15:07:57.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 1 -- 10.20.193.90:0/4096300294 >> v1:10.20.193.189:6800/3647375982 conn(0x7fe19c07a120 legacy=0x7fe19c09a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 10 monclient: shutdown 2026-02-25T15:07:57.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:57.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 1 -- 10.20.193.90:0/4096300294 >> v1:10.20.193.90:6789/0 conn(0x7fe1cc1a5660 legacy=0x7fe1cc123c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d0caa640 1 -- 10.20.193.90:0/4096300294 reap_dead start 2026-02-25T15:07:57.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 1 -- 10.20.193.90:0/4096300294 shutdown_connections 2026-02-25T15:07:57.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 1 -- 10.20.193.90:0/4096300294 >> 10.20.193.90:0/4096300294 conn(0x7fe1cc0b2f60 msgr2=0x7fe1cc15d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:57.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 1 -- 10.20.193.90:0/4096300294 shutdown_connections 2026-02-25T15:07:57.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 1 -- 10.20.193.90:0/4096300294 wait complete. 2026-02-25T15:07:57.445 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.440+0000 7fe1d2734640 1 librados: shutdown 2026-02-25T15:07:57.450 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 13 2026-02-25T15:07:57.519 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.518+0000 7f932c783640 10 monclient: get_monmap_and_config 2026-02-25T15:07:57.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.518+0000 7f932c783640 10 monclient: build_initial_monmap 2026-02-25T15:07:57.520 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.518+0000 7f932c783640 10 monclient: monmap: 2026-02-25T15:07:57.520 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:57.520 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.520 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:57.518868+0000 2026-02-25T15:07:57.520 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:57.518868+0000 2026-02-25T15:07:57.520 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:57.520 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.520 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:57.521 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:57.521 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:57.521 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.518+0000 7f932c783640 1 Processor -- start 2026-02-25T15:07:57.523 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.518+0000 7f932c783640 1 -- start start 2026-02-25T15:07:57.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.518+0000 7f932c783640 10 monclient: init 2026-02-25T15:07:57.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.519+0000 7f932c783640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:57.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.519+0000 7f932c783640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:07:57.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.519+0000 7f932c783640 10 monclient(hunting): picked mon.noname-a con 0x7f9324157780 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:57.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.519+0000 7f932c783640 10 monclient(hunting): picked mon.noname-c con 0x7f9324157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:57.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.519+0000 7f932c783640 10 monclient(hunting): picked mon.noname-b con 0x7f9324164d00 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:57.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.519+0000 7f932c783640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932405de70 con 0x7f9324157780 2026-02-25T15:07:57.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.519+0000 7f932c783640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9324055b20 con 0x7f9324164d00 2026-02-25T15:07:57.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.519+0000 7f932c783640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9324055e00 con 0x7f9324157b50 2026-02-25T15:07:57.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.519+0000 7f932c783640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:57.524 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.519+0000 7f932c783640 10 monclient(hunting): authenticate will time out at 3367.119383s 2026-02-25T15:07:57.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.519+0000 7f932acf9640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f9324164d00 0x7f93241c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45686/0 (socket says 10.20.193.90:45686) 2026-02-25T15:07:57.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.519+0000 7f932acf9640 1 -- 10.20.193.90:0/3379741533 learned_addr learned my addr 10.20.193.90:0/3379741533 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:57.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.519+0000 7f932a4f8640 1 --1- 10.20.193.90:0/3379741533 >> v1:10.20.193.90:6789/0 conn(0x7f9324157780 0x7f9324056ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39870/0 (socket says 10.20.193.90:39870) 2026-02-25T15:07:57.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 1 -- 10.20.193.90:0/3379741533 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3457051436 0 0) 0x7f9324055b20 con 0x7f9324164d00 2026-02-25T15:07:57.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 10 monclient(hunting): my global_id is 27505 2026-02-25T15:07:57.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 1 -- 10.20.193.90:0/3379741533 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f930c003610 con 0x7f9324164d00 2026-02-25T15:07:57.525 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 1 -- 10.20.193.90:0/3379741533 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2573022422 0 0) 0x7f932405de70 con 0x7f9324157780 2026-02-25T15:07:57.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 10 monclient(hunting): my global_id is 30873 2026-02-25T15:07:57.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 1 -- 10.20.193.90:0/3379741533 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9324055b20 con 0x7f9324157780 2026-02-25T15:07:57.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 1 -- 10.20.193.90:0/3379741533 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 801999936 0 0) 0x7f9324055e00 con 0x7f9324157b50 2026-02-25T15:07:57.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 10 monclient(hunting): my global_id is 28643 2026-02-25T15:07:57.526 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.521+0000 7f93294f6640 1 -- 10.20.193.90:0/3379741533 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f932405de70 con 0x7f9324157b50 2026-02-25T15:07:57.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f93294f6640 1 -- 10.20.193.90:0/3379741533 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 95742813 0 0) 0x7f930c003610 con 0x7f9324164d00 2026-02-25T15:07:57.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f93294f6640 1 -- 10.20.193.90:0/3379741533 >> v1:10.20.193.94:6789/0 conn(0x7f9324157b50 legacy=0x7f93241645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f93294f6640 1 -- 10.20.193.90:0/3379741533 >> v1:10.20.193.90:6789/0 conn(0x7f9324157780 legacy=0x7f9324056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.534 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f93294f6640 10 monclient: _finish_hunting 0 2026-02-25T15:07:57.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f93294f6640 1 monclient: found mon.noname-b 2026-02-25T15:07:57.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f93294f6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:57.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f93294f6640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:07:57.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f93294f6640 1 -- 10.20.193.90:0/3379741533 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93240560e0 con 0x7f9324164d00 2026-02-25T15:07:57.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f93294f6640 10 monclient: _finish_auth 0 2026-02-25T15:07:57.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f93294f6640 10 monclient: _check_auth_tickets 2026-02-25T15:07:57.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f93294f6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:57.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f932c783640 5 monclient: authenticate success, global_id 27505 2026-02-25T15:07:57.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f932c783640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:57.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f93294f6640 1 -- 10.20.193.90:0/3379741533 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f9320003170 con 0x7f9324164d00 2026-02-25T15:07:57.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f93294f6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:57.535 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f93294f6640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:57.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f93294f6640 10 monclient: dump: 2026-02-25T15:07:57.536 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.536 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.536 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.536 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.536 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.536 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.536 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.536 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.536 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.537 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.522+0000 7f932c783640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:57.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.523+0000 7f93294f6640 1 -- 10.20.193.90:0/3379741533 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f93200033c0 con 0x7f9324164d00 2026-02-25T15:07:57.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.523+0000 7f93294f6640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:57.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.523+0000 7f93294f6640 1 -- 10.20.193.90:0/3379741533 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f93200037c0 con 0x7f9324164d00 2026-02-25T15:07:57.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.523+0000 7f93294f6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:57.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.523+0000 7f93294f6640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:57.537 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.523+0000 7f93294f6640 10 monclient: dump: 2026-02-25T15:07:57.537 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.537 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.537 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.538 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.538 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.538 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.538 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.538 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.538 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.538 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.523+0000 7f932c783640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:57.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.523+0000 7f932c783640 4 set_mon_vals no callback set 2026-02-25T15:07:57.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 10 monclient: shutdown 2026-02-25T15:07:57.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:57.538 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 1 -- 10.20.193.90:0/3379741533 >> v1:10.20.193.93:6789/0 conn(0x7f9324164d00 legacy=0x7f93241c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 1 -- 10.20.193.90:0/3379741533 shutdown_connections 2026-02-25T15:07:57.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 1 -- 10.20.193.90:0/3379741533 >> 10.20.193.90:0/3379741533 conn(0x7f93240b2ef0 msgr2=0x7f93240b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:57.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 1 -- 10.20.193.90:0/3379741533 shutdown_connections 2026-02-25T15:07:57.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 1 -- 10.20.193.90:0/3379741533 wait complete. 2026-02-25T15:07:57.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 10 monclient: build_initial_monmap 2026-02-25T15:07:57.539 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 10 monclient: monmap: 2026-02-25T15:07:57.539 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:57.539 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:57.539 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:57.539 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:57.539 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:57.539 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:57.540 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:57.540 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:57.540 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:57.540 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 1 librados: starting msgr at 2026-02-25T15:07:57.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 1 librados: starting objecter 2026-02-25T15:07:57.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:57.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 1 Processor -- start 2026-02-25T15:07:57.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 1 -- start start 2026-02-25T15:07:57.540 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 1 librados: setting wanted keys 2026-02-25T15:07:57.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 1 librados: calling monclient init 2026-02-25T15:07:57.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.524+0000 7f932c783640 10 monclient: init 2026-02-25T15:07:57.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f932c783640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:57.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f932c783640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:07:57.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f932c783640 10 monclient(hunting): picked mon.noname-c con 0x7f9324157780 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:57.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f932c783640 10 monclient(hunting): picked mon.noname-b con 0x7f9324157b50 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:57.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f932c783640 10 monclient(hunting): picked mon.noname-a con 0x7f9324164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:57.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f932c783640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932405c280 con 0x7f9324164d00 2026-02-25T15:07:57.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f932c783640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932405c560 con 0x7f9324157b50 2026-02-25T15:07:57.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f932c783640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932405c840 con 0x7f9324157780 2026-02-25T15:07:57.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f932c783640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:57.541 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f932c783640 10 monclient(hunting): authenticate will time out at 3367.125223s 2026-02-25T15:07:57.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f932acf9640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f9324164d00 0x7f9324160fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39874/0 (socket says 10.20.193.90:39874) 2026-02-25T15:07:57.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f932acf9640 1 -- 10.20.193.90:0/3517675458 learned_addr learned my addr 10.20.193.90:0/3517675458 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:57.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f930affd640 10 client.?.objecter ms_handle_connect 0x7f9324164d00 2026-02-25T15:07:57.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f930affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:57.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f930affd640 10 client.?.objecter ms_handle_connect 0x7f9324157b50 2026-02-25T15:07:57.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f930affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:57.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f930affd640 10 client.?.objecter ms_handle_connect 0x7f9324157780 2026-02-25T15:07:57.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.525+0000 7f930affd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:57.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4100557494 0 0) 0x7f932405c280 con 0x7f9324164d00 2026-02-25T15:07:57.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.542 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 10 monclient(hunting): my global_id is 30876 2026-02-25T15:07:57.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92f4003610 con 0x7f9324164d00 2026-02-25T15:07:57.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1561794092 0 0) 0x7f932405c560 con 0x7f9324157b50 2026-02-25T15:07:57.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 10 monclient(hunting): my global_id is 27508 2026-02-25T15:07:57.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f932405c280 con 0x7f9324157b50 2026-02-25T15:07:57.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 781436110 0 0) 0x7f932405c840 con 0x7f9324157780 2026-02-25T15:07:57.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 10 monclient(hunting): my global_id is 28646 2026-02-25T15:07:57.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.526+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f932405c560 con 0x7f9324157780 2026-02-25T15:07:57.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.528+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4285647886 0 0) 0x7f932405c280 con 0x7f9324157b50 2026-02-25T15:07:57.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.528+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f932405c840 con 0x7f9324157b50 2026-02-25T15:07:57.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.528+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f9318003060 con 0x7f9324157b50 2026-02-25T15:07:57.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.528+0000 7f930affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:57.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.528+0000 7f930affd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:57.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.528+0000 7f930affd640 10 monclient(hunting): dump: 2026-02-25T15:07:57.545 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.545 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.545 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.545 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.545 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.545 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.545 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.545 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.545 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.545 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.528+0000 7f930affd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:57.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.528+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2290954903 0 0) 0x7f92f4003610 con 0x7f9324164d00 2026-02-25T15:07:57.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.528+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f932405c280 con 0x7f9324164d00 2026-02-25T15:07:57.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.528+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f9320003710 con 0x7f9324164d00 2026-02-25T15:07:57.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.528+0000 7f930affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:57.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.528+0000 7f930affd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:57.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.528+0000 7f930affd640 10 monclient(hunting): dump: 2026-02-25T15:07:57.546 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.546 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.546 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.546 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.546 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.547 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.547 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.547 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.547 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.547 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.529+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 139521551 0 0) 0x7f932405c560 con 0x7f9324157780 2026-02-25T15:07:57.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.529+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f92f4003610 con 0x7f9324157780 2026-02-25T15:07:57.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.529+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f93140030b0 con 0x7f9324157780 2026-02-25T15:07:57.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.529+0000 7f930affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:57.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.529+0000 7f930affd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:57.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.529+0000 7f930affd640 10 monclient(hunting): dump: 2026-02-25T15:07:57.548 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.548 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.548 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.548 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.548 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.548 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.548 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.548 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.548 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.548 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1586809945 0 0) 0x7f932405c840 con 0x7f9324157b50 2026-02-25T15:07:57.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 >> v1:10.20.193.94:6789/0 conn(0x7f9324157780 legacy=0x7f9324146e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 >> v1:10.20.193.90:6789/0 conn(0x7f9324164d00 legacy=0x7f9324160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f930affd640 10 monclient: _finish_hunting 0 2026-02-25T15:07:57.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f930affd640 1 monclient: found mon.b 2026-02-25T15:07:57.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f930affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:57.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f930affd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:57.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932405cb20 con 0x7f9324157b50 2026-02-25T15:07:57.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f930affd640 10 monclient: _finish_auth 0 2026-02-25T15:07:57.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f930affd640 10 monclient: _check_auth_tickets 2026-02-25T15:07:57.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f930affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:57.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f932c783640 5 monclient: authenticate success, global_id 27508 2026-02-25T15:07:57.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f932c783640 10 monclient: _renew_subs 2026-02-25T15:07:57.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f932c783640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:57.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f932c783640 1 -- 10.20.193.90:0/3517675458 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f932405ce00 con 0x7f9324157b50 2026-02-25T15:07:57.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f932c783640 10 client.27508.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:57.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f932c783640 10 monclient: _renew_subs 2026-02-25T15:07:57.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f932c783640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:57.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f932c783640 1 -- 10.20.193.90:0/3517675458 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f93241616d0 con 0x7f9324157b50 2026-02-25T15:07:57.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.530+0000 7f932c783640 1 librados: init done 2026-02-25T15:07:57.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.531+0000 7f932c783640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:57.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.531+0000 7f932c783640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:57.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.531+0000 7f932c783640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:57.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.531+0000 7f932c783640 1 -- 10.20.193.90:0/3517675458 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9324135670 con 0x7f9324157b50 2026-02-25T15:07:57.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.538+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9318003200 con 0x7f9324157b50 2026-02-25T15:07:57.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.538+0000 7f930affd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:57.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.538+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f9318003920 con 0x7f9324157b50 2026-02-25T15:07:57.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.538+0000 7f930affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:57.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.538+0000 7f930affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:57.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.538+0000 7f930affd640 10 monclient: dump: 2026-02-25T15:07:57.552 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.552 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.552 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.553 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.553 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.553 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.553 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.553 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.553 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.553 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.538+0000 7f9328cf5640 4 set_mon_vals no callback set 2026-02-25T15:07:57.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.539+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f931801ce00 con 0x7f9324157b50 2026-02-25T15:07:57.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.540+0000 7f932a4f8640 10 monclient: get_auth_request con 0x7f92f407a0d0 auth_method 0 2026-02-25T15:07:57.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.540+0000 7f930affd640 10 client.27508.objecter ms_handle_connect 0x7f92f407a0d0 2026-02-25T15:07:57.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.540+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7f9318094850 con 0x7f9324157b50 2026-02-25T15:07:57.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.540+0000 7f930affd640 10 client.27508.objecter ms_dispatch2 0x7f9324000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:57.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.540+0000 7f930affd640 3 client.27508.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:57.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.540+0000 7f930affd640 3 client.27508.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:57.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.540+0000 7f930affd640 20 client.27508.objecter dump_active .. 0 homeless 2026-02-25T15:07:57.627 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.626+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f931805e1b0 con 0x7f9324157b50 2026-02-25T15:07:57.627 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.626+0000 7f930affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:57.628 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.626+0000 7f930affd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:57.733 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.731+0000 7f932c783640 1 -- 10.20.193.90:0/3517675458 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "13", "target": ["mon-mgr", ""]}) -- 0x7f932415b3f0 con 0x7f92f407a0d0 2026-02-25T15:07:57.733 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.731+0000 7f930affd640 1 -- 10.20.193.90:0/3517675458 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 13 to deep-scrub) ==== 42+0+0 (unknown 1087616222 0 0) 0x7f932415b3f0 con 0x7f92f407a0d0 2026-02-25T15:07:57.733 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 13 to deep-scrub 2026-02-25T15:07:57.735 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.732+0000 7f932c783640 10 librados: watch_flush enter 2026-02-25T15:07:57.735 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.732+0000 7f932c783640 10 librados: watch_flush exit 2026-02-25T15:07:57.735 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.732+0000 7f932c783640 20 client.27508.objecter shutdown clearing up homeless session... 2026-02-25T15:07:57.735 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.732+0000 7f932c783640 10 client.27508.objecter successfully canceled tick 2026-02-25T15:07:57.735 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.732+0000 7f932c783640 1 -- 10.20.193.90:0/3517675458 >> v1:10.20.193.189:6800/3647375982 conn(0x7f92f407a0d0 legacy=0x7f92f409a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.735 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.732+0000 7f932c783640 10 monclient: shutdown 2026-02-25T15:07:57.735 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.732+0000 7f932c783640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:57.735 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.732+0000 7f932c783640 1 -- 10.20.193.90:0/3517675458 >> v1:10.20.193.93:6789/0 conn(0x7f9324157b50 legacy=0x7f9324133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.735 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.732+0000 7f932c783640 1 -- 10.20.193.90:0/3517675458 shutdown_connections 2026-02-25T15:07:57.736 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.732+0000 7f932c783640 1 -- 10.20.193.90:0/3517675458 >> 10.20.193.90:0/3517675458 conn(0x7f93240b2ef0 msgr2=0x7f93240b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:57.736 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.732+0000 7f932c783640 1 -- 10.20.193.90:0/3517675458 shutdown_connections 2026-02-25T15:07:57.736 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.732+0000 7f932c783640 1 -- 10.20.193.90:0/3517675458 wait complete. 2026-02-25T15:07:57.736 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.732+0000 7f932c783640 1 librados: shutdown 2026-02-25T15:07:57.742 INFO:tasks.ceph:Scrubbing osd.2 2026-02-25T15:07:57.742 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:07:57.811 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.809+0000 7f3f1d398640 10 monclient: get_monmap_and_config 2026-02-25T15:07:57.811 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.809+0000 7f3f1d398640 10 monclient: build_initial_monmap 2026-02-25T15:07:57.811 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.809+0000 7f3f1d398640 10 monclient: monmap: 2026-02-25T15:07:57.811 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:57.811 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.812 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:57.810012+0000 2026-02-25T15:07:57.812 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:57.810012+0000 2026-02-25T15:07:57.812 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:57.812 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.812 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:57.812 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:57.812 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:57.812 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.812 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.809+0000 7f3f1d398640 1 Processor -- start 2026-02-25T15:07:57.812 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.809+0000 7f3f1d398640 1 -- start start 2026-02-25T15:07:57.813 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.809+0000 7f3f1d398640 10 monclient: init 2026-02-25T15:07:57.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.810+0000 7f3f1d398640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:57.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.810+0000 7f3f1d398640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:07:57.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.810+0000 7f3f1d398640 10 monclient(hunting): picked mon.noname-c con 0x7f3f1815e2a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:57.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.810+0000 7f3f1d398640 10 monclient(hunting): picked mon.noname-a con 0x7f3f18059e10 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:57.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.810+0000 7f3f1d398640 10 monclient(hunting): picked mon.noname-b con 0x7f3f181a5660 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:57.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.810+0000 7f3f1d398640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f18162df0 con 0x7f3f18059e10 2026-02-25T15:07:57.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.810+0000 7f3f1d398640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f181630d0 con 0x7f3f181a5660 2026-02-25T15:07:57.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.810+0000 7f3f1d398640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f181633b0 con 0x7f3f1815e2a0 2026-02-25T15:07:57.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.810+0000 7f3f1d398640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:57.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.810+0000 7f3f1d398640 10 monclient(hunting): authenticate will time out at 3367.410392s 2026-02-25T15:07:57.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.810+0000 7f3f167fc640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f3f18059e10 0x7f3f1815ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39882/0 (socket says 10.20.193.90:39882) 2026-02-25T15:07:57.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.810+0000 7f3f167fc640 1 -- 10.20.193.90:0/2311424638 learned_addr learned my addr 10.20.193.90:0/2311424638 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:57.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.811+0000 7f3f15ffb640 1 -- 10.20.193.90:0/2311424638 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1295907768 0 0) 0x7f3f18162df0 con 0x7f3f18059e10 2026-02-25T15:07:57.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.812+0000 7f3f15ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.812+0000 7f3f15ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.812+0000 7f3f15ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.812+0000 7f3f15ffb640 10 monclient(hunting): my global_id is 30879 2026-02-25T15:07:57.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.812+0000 7f3f15ffb640 1 -- 10.20.193.90:0/2311424638 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f04003610 con 0x7f3f18059e10 2026-02-25T15:07:57.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.812+0000 7f3f15ffb640 1 -- 10.20.193.90:0/2311424638 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3523455584 0 0) 0x7f3f181630d0 con 0x7f3f181a5660 2026-02-25T15:07:57.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.812+0000 7f3f15ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.812+0000 7f3f15ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.812+0000 7f3f15ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.812+0000 7f3f15ffb640 10 monclient(hunting): my global_id is 27511 2026-02-25T15:07:57.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.812+0000 7f3f15ffb640 1 -- 10.20.193.90:0/2311424638 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f18162df0 con 0x7f3f181a5660 2026-02-25T15:07:57.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.813+0000 7f3f15ffb640 1 -- 10.20.193.90:0/2311424638 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1632882302 0 0) 0x7f3f181633b0 con 0x7f3f1815e2a0 2026-02-25T15:07:57.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.813+0000 7f3f15ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.813+0000 7f3f15ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.813+0000 7f3f15ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.813+0000 7f3f15ffb640 10 monclient(hunting): my global_id is 28649 2026-02-25T15:07:57.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.813+0000 7f3f15ffb640 1 -- 10.20.193.90:0/2311424638 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f181630d0 con 0x7f3f1815e2a0 2026-02-25T15:07:57.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f15ffb640 1 -- 10.20.193.90:0/2311424638 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3598412468 0 0) 0x7f3f18162df0 con 0x7f3f181a5660 2026-02-25T15:07:57.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f15ffb640 1 -- 10.20.193.90:0/2311424638 >> v1:10.20.193.94:6789/0 conn(0x7f3f1815e2a0 legacy=0x7f3f1805da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f15ffb640 1 -- 10.20.193.90:0/2311424638 >> v1:10.20.193.90:6789/0 conn(0x7f3f18059e10 legacy=0x7f3f1815ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f15ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:07:57.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f15ffb640 1 monclient: found mon.noname-b 2026-02-25T15:07:57.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f15ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:57.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f15ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:07:57.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f15ffb640 1 -- 10.20.193.90:0/2311424638 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f18163690 con 0x7f3f181a5660 2026-02-25T15:07:57.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f15ffb640 10 monclient: _finish_auth 0 2026-02-25T15:07:57.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f15ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:07:57.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f15ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:57.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f15ffb640 1 -- 10.20.193.90:0/2311424638 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3f08003170 con 0x7f3f181a5660 2026-02-25T15:07:57.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f15ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:57.824 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f15ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:57.824 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f15ffb640 10 monclient: dump: 2026-02-25T15:07:57.824 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.824 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.824 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.824 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.824 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.824 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.824 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.824 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.825 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.825 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f1d398640 5 monclient: authenticate success, global_id 27511 2026-02-25T15:07:57.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.814+0000 7f3f1d398640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:57.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.816+0000 7f3f15ffb640 1 -- 10.20.193.90:0/2311424638 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3f08003340 con 0x7f3f181a5660 2026-02-25T15:07:57.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.816+0000 7f3f15ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:57.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.816+0000 7f3f1d398640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:57.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.816+0000 7f3f1d398640 4 set_mon_vals no callback set 2026-02-25T15:07:57.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.816+0000 7f3f15ffb640 1 -- 10.20.193.90:0/2311424638 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3f08003780 con 0x7f3f181a5660 2026-02-25T15:07:57.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.816+0000 7f3f15ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:57.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.816+0000 7f3f15ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:57.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.816+0000 7f3f15ffb640 10 monclient: dump: 2026-02-25T15:07:57.826 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.826 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.826 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.826 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.826 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.826 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.826 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.826 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.826 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.826 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.816+0000 7f3f1d398640 10 monclient: shutdown 2026-02-25T15:07:57.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.816+0000 7f3f1d398640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:57.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.816+0000 7f3f1d398640 1 -- 10.20.193.90:0/2311424638 >> v1:10.20.193.93:6789/0 conn(0x7f3f181a5660 legacy=0x7f3f181c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.816+0000 7f3f1d398640 1 -- 10.20.193.90:0/2311424638 shutdown_connections 2026-02-25T15:07:57.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.816+0000 7f3f1d398640 1 -- 10.20.193.90:0/2311424638 >> 10.20.193.90:0/2311424638 conn(0x7f3f180b2f60 msgr2=0x7f3f180b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:57.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.816+0000 7f3f1d398640 1 -- 10.20.193.90:0/2311424638 shutdown_connections 2026-02-25T15:07:57.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.816+0000 7f3f1d398640 1 -- 10.20.193.90:0/2311424638 wait complete. 2026-02-25T15:07:57.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 10 monclient: build_initial_monmap 2026-02-25T15:07:57.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 10 monclient: monmap: 2026-02-25T15:07:57.827 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:57.827 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:57.828 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:57.828 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:57.828 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:57.828 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:57.828 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:57.828 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:57.828 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:57.828 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 1 librados: starting msgr at 2026-02-25T15:07:57.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 1 librados: starting objecter 2026-02-25T15:07:57.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:57.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 1 Processor -- start 2026-02-25T15:07:57.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 1 -- start start 2026-02-25T15:07:57.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 1 librados: setting wanted keys 2026-02-25T15:07:57.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 1 librados: calling monclient init 2026-02-25T15:07:57.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 10 monclient: init 2026-02-25T15:07:57.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:57.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:07:57.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 10 monclient(hunting): picked mon.noname-c con 0x7f3f18059e10 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:57.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 10 monclient(hunting): picked mon.noname-a con 0x7f3f1815e2a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:57.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 10 monclient(hunting): picked mon.noname-b con 0x7f3f181a5660 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:57.829 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.817+0000 7f3f1d398640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1812a3f0 con 0x7f3f1815e2a0 2026-02-25T15:07:57.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.818+0000 7f3f1d398640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1812a6d0 con 0x7f3f181a5660 2026-02-25T15:07:57.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.818+0000 7f3f1d398640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1812a9b0 con 0x7f3f18059e10 2026-02-25T15:07:57.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.818+0000 7f3f1d398640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:57.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.818+0000 7f3f1d398640 10 monclient(hunting): authenticate will time out at 3367.418013s 2026-02-25T15:07:57.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.818+0000 7f3f167fc640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f3f1815e2a0 0x7f3f18194740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39898/0 (socket says 10.20.193.90:39898) 2026-02-25T15:07:57.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.818+0000 7f3f167fc640 1 -- 10.20.193.90:0/1652434601 learned_addr learned my addr 10.20.193.90:0/1652434601 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:57.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.818+0000 7f3ef77fe640 10 client.?.objecter ms_handle_connect 0x7f3f1815e2a0 2026-02-25T15:07:57.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.818+0000 7f3ef77fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:57.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.818+0000 7f3ef77fe640 10 client.?.objecter ms_handle_connect 0x7f3f18059e10 2026-02-25T15:07:57.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.818+0000 7f3ef77fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:57.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.819+0000 7f3ef77fe640 10 client.?.objecter ms_handle_connect 0x7f3f181a5660 2026-02-25T15:07:57.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.819+0000 7f3ef77fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:57.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.819+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2564952287 0 0) 0x7f3f1812a3f0 con 0x7f3f1815e2a0 2026-02-25T15:07:57.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.819+0000 7f3ef77fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.819+0000 7f3ef77fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.819+0000 7f3ef77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.819+0000 7f3ef77fe640 10 monclient(hunting): my global_id is 30882 2026-02-25T15:07:57.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.819+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ee4003610 con 0x7f3f1815e2a0 2026-02-25T15:07:57.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.819+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2844242525 0 0) 0x7f3f1812a9b0 con 0x7f3f18059e10 2026-02-25T15:07:57.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.819+0000 7f3ef77fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.819+0000 7f3ef77fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.819+0000 7f3ef77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.819+0000 7f3ef77fe640 10 monclient(hunting): my global_id is 28652 2026-02-25T15:07:57.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.819+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f1812a3f0 con 0x7f3f18059e10 2026-02-25T15:07:57.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.820+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2525238078 0 0) 0x7f3f1812a6d0 con 0x7f3f181a5660 2026-02-25T15:07:57.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.820+0000 7f3ef77fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.820+0000 7f3ef77fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.820+0000 7f3ef77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.820+0000 7f3ef77fe640 10 monclient(hunting): my global_id is 27514 2026-02-25T15:07:57.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.820+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f1812a9b0 con 0x7f3f181a5660 2026-02-25T15:07:57.837 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2528299635 0 0) 0x7f3ee4003610 con 0x7f3f1815e2a0 2026-02-25T15:07:57.838 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3f1812a6d0 con 0x7f3f1815e2a0 2026-02-25T15:07:57.838 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3f00002f90 con 0x7f3f1815e2a0 2026-02-25T15:07:57.838 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:57.838 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:57.838 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 10 monclient(hunting): dump: 2026-02-25T15:07:57.838 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.838 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.838 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.838 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.838 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.839 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.839 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.839 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.839 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.839 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:57.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2402023872 0 0) 0x7f3f1812a3f0 con 0x7f3f18059e10 2026-02-25T15:07:57.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3ee4003610 con 0x7f3f18059e10 2026-02-25T15:07:57.839 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3f0c0030d0 con 0x7f3f18059e10 2026-02-25T15:07:57.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:57.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:57.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 10 monclient(hunting): dump: 2026-02-25T15:07:57.840 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.840 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.840 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.840 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.840 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.840 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.840 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.840 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.841 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.841 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3011560600 0 0) 0x7f3f1812a9b0 con 0x7f3f181a5660 2026-02-25T15:07:57.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3f1812a3f0 con 0x7f3f181a5660 2026-02-25T15:07:57.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3f080034f0 con 0x7f3f181a5660 2026-02-25T15:07:57.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:57.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:57.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.822+0000 7f3ef77fe640 10 monclient(hunting): dump: 2026-02-25T15:07:57.841 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.841 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.841 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.842 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.842 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.842 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.842 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.842 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.842 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.842 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3486714314 0 0) 0x7f3f1812a6d0 con 0x7f3f1815e2a0 2026-02-25T15:07:57.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 >> v1:10.20.193.94:6789/0 conn(0x7f3f18059e10 legacy=0x7f3f181a4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 >> v1:10.20.193.93:6789/0 conn(0x7f3f181a5660 legacy=0x7f3f18058c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3ef77fe640 10 monclient: _finish_hunting 0 2026-02-25T15:07:57.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3ef77fe640 1 monclient: found mon.a 2026-02-25T15:07:57.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3ef77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:57.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3ef77fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1812ac90 con 0x7f3f1815e2a0 2026-02-25T15:07:57.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3ef77fe640 10 monclient: _finish_auth 0 2026-02-25T15:07:57.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3ef77fe640 10 monclient: _check_auth_tickets 2026-02-25T15:07:57.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3ef77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:57.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3f1d398640 5 monclient: authenticate success, global_id 30882 2026-02-25T15:07:57.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3f1d398640 10 monclient: _renew_subs 2026-02-25T15:07:57.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3f1d398640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3f1d398640 1 -- 10.20.193.90:0/1652434601 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1812af70 con 0x7f3f1815e2a0 2026-02-25T15:07:57.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3f1d398640 10 client.30882.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:57.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3f1d398640 10 monclient: _renew_subs 2026-02-25T15:07:57.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3f1d398640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3f1d398640 1 -- 10.20.193.90:0/1652434601 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f3f18194e50 con 0x7f3f1815e2a0 2026-02-25T15:07:57.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.823+0000 7f3f1d398640 1 librados: init done 2026-02-25T15:07:57.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.824+0000 7f3f1d398640 10 client.30882.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:57.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.824+0000 7f3f1d398640 15 client.30882.objecter _session_command_op_assign -1 1 2026-02-25T15:07:57.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.824+0000 7f3f1d398640 20 client.30882.objecter _get_session osd=-1 returning homeless 2026-02-25T15:07:57.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.824+0000 7f3f1d398640 10 client.30882.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:57.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.824+0000 7f3f1d398640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.824+0000 7f3f1d398640 1 -- 10.20.193.90:0/1652434601 --> v1:10.20.193.90:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f3f1805de20 con 0x7f3f1815e2a0 2026-02-25T15:07:57.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.832+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3f1805de20 con 0x7f3f1815e2a0 2026-02-25T15:07:57.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.832+0000 7f3ef77fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:57.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.832+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f3f00004a70 con 0x7f3f1815e2a0 2026-02-25T15:07:57.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.832+0000 7f3ef77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:57.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.832+0000 7f3ef77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:57.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.832+0000 7f3ef77fe640 10 monclient: dump: 2026-02-25T15:07:57.845 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.845 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.845 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.845 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.845 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.845 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.845 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.845 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.845 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.846 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.832+0000 7f3f157fa640 4 set_mon_vals no callback set 2026-02-25T15:07:57.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.833+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f3f0001df50 con 0x7f3f1815e2a0 2026-02-25T15:07:57.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.834+0000 7f3f16ffd640 10 monclient: get_auth_request con 0x7f3ee407a0f0 auth_method 0 2026-02-25T15:07:57.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.834+0000 7f3ef77fe640 10 client.30882.objecter ms_handle_connect 0x7f3ee407a0f0 2026-02-25T15:07:57.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7f3f00095880 con 0x7f3f1815e2a0 2026-02-25T15:07:57.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 10 client.30882.objecter ms_dispatch2 0x7f3f18000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:57.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 3 client.30882.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:57.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 3 client.30882.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:57.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 10 client.30882.objecter checking command 1 2026-02-25T15:07:57.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 20 client.30882.objecter _get_session s=0x7f3ee40a43a0 osd=2 3 2026-02-25T15:07:57.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 20 client.30882.objecter put_session s=0x7f3ee40a43a0 osd=2 3 2026-02-25T15:07:57.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 15 client.30882.objecter _session_command_op_remove -1 1 2026-02-25T15:07:57.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 20 client.30882.objecter _get_session s=0x7f3ee40a43a0 osd=2 3 2026-02-25T15:07:57.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 20 client.30882.objecter get_session s=0x7f3ee40a43a0 osd=2 3 2026-02-25T15:07:57.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 15 client.30882.objecter _session_command_op_assign 2 1 2026-02-25T15:07:57.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 20 client.30882.objecter put_session s=0x7f3ee40a43a0 osd=2 4 2026-02-25T15:07:57.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 10 client.30882.objecter _send_command 1 2026-02-25T15:07:57.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 --> v1:10.20.193.94:6800/3705139403 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ee40c66b0 con 0x7f3ee40a4a50 2026-02-25T15:07:57.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 20 client.30882.objecter dump_active .. 0 homeless 2026-02-25T15:07:57.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_get_version_reply(handle=1 version=338) ==== 24+0+0 (unknown 1001891302 0 0) 0x7f3f0005ae50 con 0x7f3f1815e2a0 2026-02-25T15:07:57.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3ef77fe640 10 monclient: handle_get_version_reply finishing 1 version 338 2026-02-25T15:07:57.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.835+0000 7f3f177fe640 10 monclient: get_auth_request con 0x7f3ee40a4a50 auth_method 0 2026-02-25T15:07:57.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.837+0000 7f3ef77fe640 10 client.30882.objecter ms_handle_connect 0x7f3ee40a4a50 2026-02-25T15:07:57.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.846+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== osd.2 v1:10.20.193.94:6800/3705139403 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f3ee40c66b0 con 0x7f3ee40a4a50 2026-02-25T15:07:57.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.846+0000 7f3ef77fe640 10 client.30882.objecter ms_dispatch2 0x7f3f18000b90 command_reply(tid 1: 0 ) 2026-02-25T15:07:57.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.846+0000 7f3ef77fe640 10 client.30882.objecter _finish_command 1 = system:0 2026-02-25T15:07:57.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.846+0000 7f3ef77fe640 20 client.30882.objecter put_session s=0x7f3ee40a43a0 osd=2 4 2026-02-25T15:07:57.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.846+0000 7f3ef77fe640 15 client.30882.objecter _session_command_op_remove 2 1 2026-02-25T15:07:57.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.853+0000 7f3f1d398640 10 client.30882.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:07:57.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.853+0000 7f3f1d398640 15 client.30882.objecter _session_command_op_assign -1 2 2026-02-25T15:07:57.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.853+0000 7f3f1d398640 20 client.30882.objecter _get_session s=0x7f3ee40a43a0 osd=2 3 2026-02-25T15:07:57.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.853+0000 7f3f1d398640 20 client.30882.objecter put_session s=0x7f3ee40a43a0 osd=2 3 2026-02-25T15:07:57.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.853+0000 7f3f1d398640 20 client.30882.objecter _get_session s=0x7f3ee40a43a0 osd=2 3 2026-02-25T15:07:57.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.853+0000 7f3f1d398640 15 client.30882.objecter _session_command_op_remove -1 2 2026-02-25T15:07:57.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.853+0000 7f3f1d398640 20 client.30882.objecter get_session s=0x7f3ee40a43a0 osd=2 3 2026-02-25T15:07:57.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.853+0000 7f3f1d398640 15 client.30882.objecter _session_command_op_assign 2 2 2026-02-25T15:07:57.855 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.853+0000 7f3f1d398640 20 client.30882.objecter put_session s=0x7f3ee40a43a0 osd=2 4 2026-02-25T15:07:57.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.853+0000 7f3f1d398640 10 client.30882.objecter _send_command 2 2026-02-25T15:07:57.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.853+0000 7f3f1d398640 1 -- 10.20.193.90:0/1652434601 --> v1:10.20.193.94:6800/3705139403 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3f1805db40 con 0x7f3ee40a4a50 2026-02-25T15:07:57.862 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3ef77fe640 1 -- 10.20.193.90:0/1652434601 <== osd.2 v1:10.20.193.94:6800/3705139403 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f3f1805db40 con 0x7f3ee40a4a50 2026-02-25T15:07:57.862 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3ef77fe640 10 client.30882.objecter ms_dispatch2 0x7f3f18000b90 command_reply(tid 2: 0 ) 2026-02-25T15:07:57.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3ef77fe640 10 client.30882.objecter _finish_command 2 = system:0 2026-02-25T15:07:57.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3ef77fe640 20 client.30882.objecter put_session s=0x7f3ee40a43a0 osd=2 4 2026-02-25T15:07:57.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3ef77fe640 15 client.30882.objecter _session_command_op_remove 2 2 2026-02-25T15:07:57.863 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:07:57.863 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-25T15:07:57.863 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:07:57.865 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 10 librados: watch_flush enter 2026-02-25T15:07:57.865 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 10 librados: watch_flush exit 2026-02-25T15:07:57.865 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 10 client.30882.objecter close_session for osd.2 2026-02-25T15:07:57.865 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 1 -- 10.20.193.90:0/1652434601 >> v1:10.20.193.94:6800/3705139403 conn(0x7f3ee40a4a50 legacy=0x7f3ee40c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.865 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 20 client.30882.objecter put_session s=0x7f3ee40a43a0 osd=2 1 2026-02-25T15:07:57.865 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 20 client.30882.objecter shutdown clearing up homeless session... 2026-02-25T15:07:57.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 10 client.30882.objecter successfully canceled tick 2026-02-25T15:07:57.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 1 -- 10.20.193.90:0/1652434601 >> v1:10.20.193.189:6800/3647375982 conn(0x7f3ee407a0f0 legacy=0x7f3ee409a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 10 monclient: shutdown 2026-02-25T15:07:57.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:57.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 1 -- 10.20.193.90:0/1652434601 >> v1:10.20.193.90:6789/0 conn(0x7f3f1815e2a0 legacy=0x7f3f18194740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f177fe640 1 -- 10.20.193.90:0/1652434601 reap_dead start 2026-02-25T15:07:57.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 1 -- 10.20.193.90:0/1652434601 shutdown_connections 2026-02-25T15:07:57.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 1 -- 10.20.193.90:0/1652434601 >> 10.20.193.90:0/1652434601 conn(0x7f3f180b2f60 msgr2=0x7f3f1815bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:57.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 1 -- 10.20.193.90:0/1652434601 shutdown_connections 2026-02-25T15:07:57.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 1 -- 10.20.193.90:0/1652434601 wait complete. 2026-02-25T15:07:57.866 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.861+0000 7f3f1d398640 1 librados: shutdown 2026-02-25T15:07:57.870 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-25T15:07:57.940 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.938+0000 7f43279c3640 10 monclient: get_monmap_and_config 2026-02-25T15:07:57.940 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.938+0000 7f43279c3640 10 monclient: build_initial_monmap 2026-02-25T15:07:57.940 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.938+0000 7f43279c3640 10 monclient: monmap: 2026-02-25T15:07:57.941 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:57.941 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.941 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:57.939591+0000 2026-02-25T15:07:57.941 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:57.939591+0000 2026-02-25T15:07:57.941 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:57.941 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.941 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:57.941 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:57.941 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:57.941 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.939+0000 7f43279c3640 1 Processor -- start 2026-02-25T15:07:57.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.939+0000 7f43279c3640 1 -- start start 2026-02-25T15:07:57.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.939+0000 7f43279c3640 10 monclient: init 2026-02-25T15:07:57.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.940+0000 7f43279c3640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:57.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.940+0000 7f43279c3640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:07:57.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.940+0000 7f43279c3640 10 monclient(hunting): picked mon.noname-a con 0x7f432015d0c0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:57.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.940+0000 7f43279c3640 10 monclient(hunting): picked mon.noname-b con 0x7f43200b0ca0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:57.944 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.940+0000 7f43279c3640 10 monclient(hunting): picked mon.noname-c con 0x7f43200b28b0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:57.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.940+0000 7f43279c3640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43200b8540 con 0x7f432015d0c0 2026-02-25T15:07:57.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.940+0000 7f43279c3640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43200b5da0 con 0x7f43200b0ca0 2026-02-25T15:07:57.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.940+0000 7f43279c3640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43200b6080 con 0x7f43200b28b0 2026-02-25T15:07:57.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.940+0000 7f43279c3640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:57.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.940+0000 7f43279c3640 10 monclient(hunting): authenticate will time out at 3367.540173s 2026-02-25T15:07:57.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.940+0000 7f4324f37640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f43200b0ca0 0x7f43200b7720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45736/0 (socket says 10.20.193.90:45736) 2026-02-25T15:07:57.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.940+0000 7f4324f37640 1 -- 10.20.193.90:0/1839415071 learned_addr learned my addr 10.20.193.90:0/1839415071 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:57.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 1 -- 10.20.193.90:0/1839415071 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 815448931 0 0) 0x7f43200b8540 con 0x7f432015d0c0 2026-02-25T15:07:57.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.945 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 10 monclient(hunting): my global_id is 30885 2026-02-25T15:07:57.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 1 -- 10.20.193.90:0/1839415071 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4308003610 con 0x7f432015d0c0 2026-02-25T15:07:57.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 1 -- 10.20.193.90:0/1839415071 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 466281676 0 0) 0x7f43200b5da0 con 0x7f43200b0ca0 2026-02-25T15:07:57.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 10 monclient(hunting): my global_id is 27517 2026-02-25T15:07:57.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 1 -- 10.20.193.90:0/1839415071 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43200b8540 con 0x7f43200b0ca0 2026-02-25T15:07:57.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 1 -- 10.20.193.90:0/1839415071 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3709112321 0 0) 0x7f43200b6080 con 0x7f43200b28b0 2026-02-25T15:07:57.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.946 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 10 monclient(hunting): my global_id is 28655 2026-02-25T15:07:57.947 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.941+0000 7f430ffff640 1 -- 10.20.193.90:0/1839415071 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43200b5da0 con 0x7f43200b28b0 2026-02-25T15:07:57.954 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f430ffff640 1 -- 10.20.193.90:0/1839415071 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3960923542 0 0) 0x7f43200b8540 con 0x7f43200b0ca0 2026-02-25T15:07:57.954 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f430ffff640 1 -- 10.20.193.90:0/1839415071 >> v1:10.20.193.94:6789/0 conn(0x7f43200b28b0 legacy=0x7f43200b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.954 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f430ffff640 1 -- 10.20.193.90:0/1839415071 >> v1:10.20.193.90:6789/0 conn(0x7f432015d0c0 legacy=0x7f43200b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.954 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f430ffff640 10 monclient: _finish_hunting 0 2026-02-25T15:07:57.954 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f430ffff640 1 monclient: found mon.noname-b 2026-02-25T15:07:57.954 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f430ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:57.954 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f430ffff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:07:57.954 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f430ffff640 1 -- 10.20.193.90:0/1839415071 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4320163e10 con 0x7f43200b0ca0 2026-02-25T15:07:57.954 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f430ffff640 10 monclient: _finish_auth 0 2026-02-25T15:07:57.954 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f430ffff640 10 monclient: _check_auth_tickets 2026-02-25T15:07:57.955 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f430ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:57.955 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f430ffff640 1 -- 10.20.193.90:0/1839415071 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4314002df0 con 0x7f43200b0ca0 2026-02-25T15:07:57.955 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f430ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:57.955 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f430ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:57.955 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f430ffff640 10 monclient: dump: 2026-02-25T15:07:57.955 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.955 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.955 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.955 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.956 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.956 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.956 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.956 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.956 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.956 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f43279c3640 5 monclient: authenticate success, global_id 27517 2026-02-25T15:07:57.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.943+0000 7f43279c3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:57.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.944+0000 7f430ffff640 1 -- 10.20.193.90:0/1839415071 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4314002fc0 con 0x7f43200b0ca0 2026-02-25T15:07:57.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.944+0000 7f430ffff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:57.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.944+0000 7f430ffff640 1 -- 10.20.193.90:0/1839415071 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4314003400 con 0x7f43200b0ca0 2026-02-25T15:07:57.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.944+0000 7f430ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:57.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.944+0000 7f430ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:57.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.944+0000 7f430ffff640 10 monclient: dump: 2026-02-25T15:07:57.957 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.957 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.957 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.957 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.957 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.957 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.957 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.957 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.957 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.957 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.944+0000 7f43279c3640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:57.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.944+0000 7f43279c3640 4 set_mon_vals no callback set 2026-02-25T15:07:57.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.944+0000 7f43279c3640 10 monclient: shutdown 2026-02-25T15:07:57.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.944+0000 7f43279c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:57.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.944+0000 7f43279c3640 1 -- 10.20.193.90:0/1839415071 >> v1:10.20.193.93:6789/0 conn(0x7f43200b0ca0 legacy=0x7f43200b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.944+0000 7f43279c3640 1 -- 10.20.193.90:0/1839415071 shutdown_connections 2026-02-25T15:07:57.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.944+0000 7f43279c3640 1 -- 10.20.193.90:0/1839415071 >> 10.20.193.90:0/1839415071 conn(0x7f43200b5290 msgr2=0x7f43200b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:57.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.944+0000 7f43279c3640 1 -- 10.20.193.90:0/1839415071 shutdown_connections 2026-02-25T15:07:57.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.944+0000 7f43279c3640 1 -- 10.20.193.90:0/1839415071 wait complete. 2026-02-25T15:07:57.959 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 10 monclient: build_initial_monmap 2026-02-25T15:07:57.959 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 10 monclient: monmap: 2026-02-25T15:07:57.959 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:57.959 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:57.959 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:57.959 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:57.959 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:57.959 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:57.959 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:57.959 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:57.959 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:57.960 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 1 librados: starting msgr at 2026-02-25T15:07:57.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 1 librados: starting objecter 2026-02-25T15:07:57.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:57.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 1 Processor -- start 2026-02-25T15:07:57.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 1 -- start start 2026-02-25T15:07:57.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 1 librados: setting wanted keys 2026-02-25T15:07:57.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 1 librados: calling monclient init 2026-02-25T15:07:57.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 10 monclient: init 2026-02-25T15:07:57.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:57.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:07:57.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 10 monclient(hunting): picked mon.noname-a con 0x7f43200b0ca0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:57.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 10 monclient(hunting): picked mon.noname-b con 0x7f43200b28b0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:57.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 10 monclient(hunting): picked mon.noname-c con 0x7f432015d0c0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:57.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4320153930 con 0x7f43200b0ca0 2026-02-25T15:07:57.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43201fc370 con 0x7f43200b28b0 2026-02-25T15:07:57.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43201fbdb0 con 0x7f432015d0c0 2026-02-25T15:07:57.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:57.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.945+0000 7f43279c3640 10 monclient(hunting): authenticate will time out at 3367.545906s 2026-02-25T15:07:57.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.946+0000 7f4324f37640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f43200b28b0 0x7f43201fb470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45738/0 (socket says 10.20.193.90:45738) 2026-02-25T15:07:57.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.946+0000 7f4324f37640 1 -- 10.20.193.90:0/1362311398 learned_addr learned my addr 10.20.193.90:0/1362311398 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:57.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.946+0000 7f430dffb640 10 client.?.objecter ms_handle_connect 0x7f432015d0c0 2026-02-25T15:07:57.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.946+0000 7f430dffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:57.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.946+0000 7f430dffb640 10 client.?.objecter ms_handle_connect 0x7f43200b28b0 2026-02-25T15:07:57.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.946+0000 7f430dffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:57.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.946+0000 7f430dffb640 10 client.?.objecter ms_handle_connect 0x7f43200b0ca0 2026-02-25T15:07:57.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.946+0000 7f430dffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:57.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1008698350 0 0) 0x7f43201fbdb0 con 0x7f432015d0c0 2026-02-25T15:07:57.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 10 monclient(hunting): my global_id is 28658 2026-02-25T15:07:57.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42f0003610 con 0x7f432015d0c0 2026-02-25T15:07:57.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 51124778 0 0) 0x7f43201fc370 con 0x7f43200b28b0 2026-02-25T15:07:57.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 10 monclient(hunting): my global_id is 27520 2026-02-25T15:07:57.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43201fbdb0 con 0x7f43200b28b0 2026-02-25T15:07:57.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 705618035 0 0) 0x7f4320153930 con 0x7f43200b0ca0 2026-02-25T15:07:57.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:57.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:57.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:57.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 10 monclient(hunting): my global_id is 30888 2026-02-25T15:07:57.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.947+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43201fc370 con 0x7f43200b0ca0 2026-02-25T15:07:57.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3307481015 0 0) 0x7f42f0003610 con 0x7f432015d0c0 2026-02-25T15:07:57.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4320153930 con 0x7f432015d0c0 2026-02-25T15:07:57.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f431c003390 con 0x7f432015d0c0 2026-02-25T15:07:57.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:57.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:07:57.964 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 10 monclient(hunting): dump: 2026-02-25T15:07:57.964 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.964 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.965 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.965 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.965 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.965 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.965 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.965 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.965 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.965 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.965 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:57.965 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2423378556 0 0) 0x7f43201fc370 con 0x7f43200b0ca0 2026-02-25T15:07:57.965 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f42f0003610 con 0x7f43200b0ca0 2026-02-25T15:07:57.966 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4310002fb0 con 0x7f43200b0ca0 2026-02-25T15:07:57.966 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:57.966 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:57.966 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 10 monclient(hunting): dump: 2026-02-25T15:07:57.966 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.966 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.966 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.966 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.966 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.966 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.966 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.967 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.967 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.967 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.967 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1585985240 0 0) 0x7f43201fbdb0 con 0x7f43200b28b0 2026-02-25T15:07:57.967 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f43201fc370 con 0x7f43200b28b0 2026-02-25T15:07:57.967 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4314003310 con 0x7f43200b28b0 2026-02-25T15:07:57.967 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:57.967 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:57.967 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.950+0000 7f430dffb640 10 monclient(hunting): dump: 2026-02-25T15:07:57.967 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:57.968 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:57.968 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.968 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:57.968 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:57.968 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:57.968 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:57.968 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:57.968 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:57.968 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:57.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 682420482 0 0) 0x7f42f0003610 con 0x7f43200b0ca0 2026-02-25T15:07:57.968 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 >> v1:10.20.193.94:6789/0 conn(0x7f432015d0c0 legacy=0x7f4320158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 >> v1:10.20.193.93:6789/0 conn(0x7f43200b28b0 legacy=0x7f43201fb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:57.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f430dffb640 10 monclient: _finish_hunting 0 2026-02-25T15:07:57.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f430dffb640 1 monclient: found mon.a 2026-02-25T15:07:57.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f430dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:57.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f430dffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43201fc090 con 0x7f43200b0ca0 2026-02-25T15:07:57.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f430dffb640 10 monclient: _finish_auth 0 2026-02-25T15:07:57.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f430dffb640 10 monclient: _check_auth_tickets 2026-02-25T15:07:57.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f430dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:57.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f43279c3640 5 monclient: authenticate success, global_id 30888 2026-02-25T15:07:57.969 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f43279c3640 10 monclient: _renew_subs 2026-02-25T15:07:57.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f43279c3640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f43279c3640 1 -- 10.20.193.90:0/1362311398 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4320138480 con 0x7f43200b0ca0 2026-02-25T15:07:57.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f43279c3640 10 client.30888.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:57.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f43279c3640 10 monclient: _renew_subs 2026-02-25T15:07:57.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f43279c3640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f43279c3640 1 -- 10.20.193.90:0/1362311398 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4320158b40 con 0x7f43200b0ca0 2026-02-25T15:07:57.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.951+0000 7f43279c3640 1 librados: init done 2026-02-25T15:07:57.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.952+0000 7f43279c3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:57.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.952+0000 7f43279c3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:57.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.952+0000 7f43279c3640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:57.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:57.952+0000 7f43279c3640 1 -- 10.20.193.90:0/1362311398 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432005e210 con 0x7f43200b0ca0 2026-02-25T15:07:58.051 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.049+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4310003700 con 0x7f43200b0ca0 2026-02-25T15:07:58.051 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.049+0000 7f430dffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:58.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.049+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f4310003a50 con 0x7f43200b0ca0 2026-02-25T15:07:58.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.049+0000 7f430dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:58.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.049+0000 7f430dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:58.052 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.049+0000 7f430dffb640 10 monclient: dump: 2026-02-25T15:07:58.052 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.052 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.052 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.052 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.052 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.052 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.052 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.053 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.053 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.053 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.049+0000 7f430f7fe640 4 set_mon_vals no callback set 2026-02-25T15:07:58.053 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.050+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f431001cf30 con 0x7f43200b0ca0 2026-02-25T15:07:58.054 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.050+0000 7f4324f37640 10 monclient: get_auth_request con 0x7f42f007a0b0 auth_method 0 2026-02-25T15:07:58.054 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.051+0000 7f430dffb640 10 client.30888.objecter ms_handle_connect 0x7f42f007a0b0 2026-02-25T15:07:58.055 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.053+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7f43100947e0 con 0x7f43200b0ca0 2026-02-25T15:07:58.055 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.053+0000 7f430dffb640 10 client.30888.objecter ms_dispatch2 0x7f4320000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:58.055 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.053+0000 7f430dffb640 3 client.30888.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:58.055 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.053+0000 7f430dffb640 3 client.30888.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:58.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.053+0000 7f430dffb640 20 client.30888.objecter dump_active .. 0 homeless 2026-02-25T15:07:58.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.053+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f431005e140 con 0x7f43200b0ca0 2026-02-25T15:07:58.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.053+0000 7f430dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:58.056 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.053+0000 7f430dffb640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:58.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.159+0000 7f43279c3640 1 -- 10.20.193.90:0/1362311398 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f43200b4c40 con 0x7f42f007a0b0 2026-02-25T15:07:58.161 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.159+0000 7f430dffb640 1 -- 10.20.193.90:0/1362311398 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (unknown 3049115190 0 0) 0x7f43200b4c40 con 0x7f42f007a0b0 2026-02-25T15:07:58.161 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 2 to deep-scrub 2026-02-25T15:07:58.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.160+0000 7f43279c3640 10 librados: watch_flush enter 2026-02-25T15:07:58.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.160+0000 7f43279c3640 10 librados: watch_flush exit 2026-02-25T15:07:58.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.160+0000 7f43279c3640 20 client.30888.objecter shutdown clearing up homeless session... 2026-02-25T15:07:58.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.160+0000 7f43279c3640 10 client.30888.objecter successfully canceled tick 2026-02-25T15:07:58.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.160+0000 7f43279c3640 1 -- 10.20.193.90:0/1362311398 >> v1:10.20.193.189:6800/3647375982 conn(0x7f42f007a0b0 legacy=0x7f42f009a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.160+0000 7f43279c3640 10 monclient: shutdown 2026-02-25T15:07:58.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.160+0000 7f43279c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:58.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.160+0000 7f43279c3640 1 -- 10.20.193.90:0/1362311398 >> v1:10.20.193.90:6789/0 conn(0x7f43200b0ca0 legacy=0x7f4320136850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.160+0000 7f43279c3640 1 -- 10.20.193.90:0/1362311398 shutdown_connections 2026-02-25T15:07:58.163 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.160+0000 7f43279c3640 1 -- 10.20.193.90:0/1362311398 >> 10.20.193.90:0/1362311398 conn(0x7f43200b5290 msgr2=0x7f4320135940 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:58.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.160+0000 7f43279c3640 1 -- 10.20.193.90:0/1362311398 shutdown_connections 2026-02-25T15:07:58.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.160+0000 7f43279c3640 1 -- 10.20.193.90:0/1362311398 wait complete. 2026-02-25T15:07:58.164 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.160+0000 7f43279c3640 1 librados: shutdown 2026-02-25T15:07:58.170 INFO:tasks.ceph:Scrubbing osd.6 2026-02-25T15:07:58.170 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:07:58.243 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.241+0000 7fedd7c3c640 10 monclient: get_monmap_and_config 2026-02-25T15:07:58.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.241+0000 7fedd7c3c640 10 monclient: build_initial_monmap 2026-02-25T15:07:58.244 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.241+0000 7fedd7c3c640 10 monclient: monmap: 2026-02-25T15:07:58.244 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:58.244 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.244 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:58.242334+0000 2026-02-25T15:07:58.244 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:58.242334+0000 2026-02-25T15:07:58.244 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:58.244 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.244 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:58.244 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:58.245 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:58.245 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.242+0000 7fedd7c3c640 1 Processor -- start 2026-02-25T15:07:58.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.242+0000 7fedd7c3c640 1 -- start start 2026-02-25T15:07:58.245 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.242+0000 7fedd7c3c640 10 monclient: init 2026-02-25T15:07:58.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.242+0000 7fedd7c3c640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:58.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.242+0000 7fedd7c3c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:07:58.246 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.242+0000 7fedd7c3c640 10 monclient(hunting): picked mon.noname-a con 0x7fedd015e2a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:58.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.242+0000 7fedd7c3c640 10 monclient(hunting): picked mon.noname-b con 0x7fedd0059e10 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:58.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.242+0000 7fedd7c3c640 10 monclient(hunting): picked mon.noname-c con 0x7fedd01a5660 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:58.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.242+0000 7fedd7c3c640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd0162df0 con 0x7fedd015e2a0 2026-02-25T15:07:58.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.242+0000 7fedd7c3c640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd01630d0 con 0x7fedd0059e10 2026-02-25T15:07:58.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.242+0000 7fedd7c3c640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd01633b0 con 0x7fedd01a5660 2026-02-25T15:07:58.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.242+0000 7fedd7c3c640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:58.247 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.242+0000 7fedd7c3c640 10 monclient(hunting): authenticate will time out at 3367.842760s 2026-02-25T15:07:58.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.251+0000 7fedd59b1640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fedd015e2a0 0x7fedd005da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39930/0 (socket says 10.20.193.90:39930) 2026-02-25T15:07:58.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.251+0000 7fedd59b1640 1 -- 10.20.193.90:0/1040518288 learned_addr learned my addr 10.20.193.90:0/1040518288 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:58.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.252+0000 7fedd49af640 1 -- 10.20.193.90:0/1040518288 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1340591603 0 0) 0x7fedd0162df0 con 0x7fedd015e2a0 2026-02-25T15:07:58.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.252+0000 7fedd49af640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.252+0000 7fedd49af640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.252+0000 7fedd49af640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.253 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.252+0000 7fedd49af640 10 monclient(hunting): my global_id is 30891 2026-02-25T15:07:58.254 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.252+0000 7fedd49af640 1 -- 10.20.193.90:0/1040518288 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedb8003610 con 0x7fedd015e2a0 2026-02-25T15:07:58.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.254+0000 7fedd49af640 1 -- 10.20.193.90:0/1040518288 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 332962565 0 0) 0x7fedd01630d0 con 0x7fedd0059e10 2026-02-25T15:07:58.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.254+0000 7fedd49af640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.254+0000 7fedd49af640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.255 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.254+0000 7fedd49af640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.254+0000 7fedd49af640 10 monclient(hunting): my global_id is 27523 2026-02-25T15:07:58.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.254+0000 7fedd49af640 1 -- 10.20.193.90:0/1040518288 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedd0162df0 con 0x7fedd0059e10 2026-02-25T15:07:58.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 1 -- 10.20.193.90:0/1040518288 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3762876971 0 0) 0x7fedd01633b0 con 0x7fedd01a5660 2026-02-25T15:07:58.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 10 monclient(hunting): my global_id is 28661 2026-02-25T15:07:58.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 1 -- 10.20.193.90:0/1040518288 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedd01630d0 con 0x7fedd01a5660 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 1 -- 10.20.193.90:0/1040518288 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2501562073 0 0) 0x7fedb8003610 con 0x7fedd015e2a0 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 1 -- 10.20.193.90:0/1040518288 >> v1:10.20.193.94:6789/0 conn(0x7fedd01a5660 legacy=0x7fedd01c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 1 -- 10.20.193.90:0/1040518288 >> v1:10.20.193.93:6789/0 conn(0x7fedd0059e10 legacy=0x7fedd015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 10 monclient: _finish_hunting 0 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 1 monclient: found mon.noname-a 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 1 -- 10.20.193.90:0/1040518288 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd0163690 con 0x7fedd015e2a0 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 10 monclient: _finish_auth 0 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 10 monclient: _check_auth_tickets 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 1 -- 10.20.193.90:0/1040518288 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fedc0002df0 con 0x7fedd015e2a0 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd49af640 10 monclient: dump: 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.257 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd7c3c640 5 monclient: authenticate success, global_id 30891 2026-02-25T15:07:58.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.255+0000 7fedd7c3c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:58.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.256+0000 7fedd49af640 1 -- 10.20.193.90:0/1040518288 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fedc0002fc0 con 0x7fedd015e2a0 2026-02-25T15:07:58.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.256+0000 7fedd49af640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:58.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.256+0000 7fedd49af640 1 -- 10.20.193.90:0/1040518288 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fedc0003400 con 0x7fedd015e2a0 2026-02-25T15:07:58.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.256+0000 7fedd49af640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.256+0000 7fedd49af640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.256+0000 7fedd49af640 10 monclient: dump: 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.256+0000 7fedd7c3c640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.256+0000 7fedd7c3c640 4 set_mon_vals no callback set 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.256+0000 7fedd7c3c640 10 monclient: shutdown 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.256+0000 7fedd7c3c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.256+0000 7fedd7c3c640 1 -- 10.20.193.90:0/1040518288 >> v1:10.20.193.90:6789/0 conn(0x7fedd015e2a0 legacy=0x7fedd005da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.256+0000 7fedd7c3c640 1 -- 10.20.193.90:0/1040518288 shutdown_connections 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.256+0000 7fedd7c3c640 1 -- 10.20.193.90:0/1040518288 >> 10.20.193.90:0/1040518288 conn(0x7fedd00b2f60 msgr2=0x7fedd00b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.256+0000 7fedd7c3c640 1 -- 10.20.193.90:0/1040518288 shutdown_connections 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.256+0000 7fedd7c3c640 1 -- 10.20.193.90:0/1040518288 wait complete. 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 10 monclient: build_initial_monmap 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 10 monclient: monmap: 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:58.259 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:58.260 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:58.260 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:58.260 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:58.260 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:58.260 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 1 librados: starting msgr at 2026-02-25T15:07:58.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 1 librados: starting objecter 2026-02-25T15:07:58.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:58.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 1 Processor -- start 2026-02-25T15:07:58.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 1 -- start start 2026-02-25T15:07:58.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 1 librados: setting wanted keys 2026-02-25T15:07:58.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 1 librados: calling monclient init 2026-02-25T15:07:58.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 10 monclient: init 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 10 monclient(hunting): picked mon.noname-a con 0x7fedd0059e10 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 10 monclient(hunting): picked mon.noname-b con 0x7fedd015e2a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 10 monclient(hunting): picked mon.noname-c con 0x7fedd01a5660 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd0193d00 con 0x7fedd0059e10 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd0193f70 con 0x7fedd015e2a0 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd0194250 con 0x7fedd01a5660 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.257+0000 7fedd7c3c640 10 monclient(hunting): authenticate will time out at 3367.857838s 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.258+0000 7fedd51b0640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fedd015e2a0 0x7fedd011e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45750/0 (socket says 10.20.193.90:45750) 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.258+0000 7fedd51b0640 1 -- 10.20.193.90:0/3742749902 learned_addr learned my addr 10.20.193.90:0/3742749902 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.258+0000 7fedbe7fc640 10 client.?.objecter ms_handle_connect 0x7fedd015e2a0 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.258+0000 7fedbe7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.258+0000 7fedbe7fc640 10 client.?.objecter ms_handle_connect 0x7fedd0059e10 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.258+0000 7fedbe7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:58.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.258+0000 7fedbe7fc640 10 client.?.objecter ms_handle_connect 0x7fedd01a5660 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.258+0000 7fedbe7fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 941433536 0 0) 0x7fedd0193d00 con 0x7fedd0059e10 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 10 monclient(hunting): my global_id is 30894 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feda0003610 con 0x7fedd0059e10 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1306004441 0 0) 0x7fedd0193f70 con 0x7fedd015e2a0 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 10 monclient(hunting): my global_id is 27526 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedd0193d00 con 0x7fedd015e2a0 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3036004389 0 0) 0x7fedd0194250 con 0x7fedd01a5660 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 10 monclient(hunting): my global_id is 28664 2026-02-25T15:07:58.262 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.259+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedd0193f70 con 0x7fedd01a5660 2026-02-25T15:07:58.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.261+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2608554937 0 0) 0x7fedd0193d00 con 0x7fedd015e2a0 2026-02-25T15:07:58.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.261+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fedd0194250 con 0x7fedd015e2a0 2026-02-25T15:07:58.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.261+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fedc40030f0 con 0x7fedd015e2a0 2026-02-25T15:07:58.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.261+0000 7fedbe7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:58.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.261+0000 7fedbe7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:58.263 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.261+0000 7fedbe7fc640 10 monclient(hunting): dump: 2026-02-25T15:07:58.263 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.263 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.263 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.263 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.263 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.263 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.263 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.263 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.263 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.261+0000 7fedbe7fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.261+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 94472208 0 0) 0x7fedd0193f70 con 0x7fedd01a5660 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.261+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fedd0193d00 con 0x7fedd01a5660 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.261+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fedcc003410 con 0x7fedd01a5660 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.261+0000 7fedbe7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.261+0000 7fedbe7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.261+0000 7fedbe7fc640 10 monclient(hunting): dump: 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.262+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3604941202 0 0) 0x7feda0003610 con 0x7fedd0059e10 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.262+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fedd0193f70 con 0x7fedd0059e10 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.262+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fedc0003310 con 0x7fedd0059e10 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.262+0000 7fedbe7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.262+0000 7fedbe7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.262+0000 7fedbe7fc640 10 monclient(hunting): dump: 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.264 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.265 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.265 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.265 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.265 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.265 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.265 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2909853995 0 0) 0x7fedd0194250 con 0x7fedd015e2a0 2026-02-25T15:07:58.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 >> v1:10.20.193.94:6789/0 conn(0x7fedd01a5660 legacy=0x7fedd0164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 >> v1:10.20.193.90:6789/0 conn(0x7fedd0059e10 legacy=0x7fedd018fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedbe7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:07:58.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedbe7fc640 1 monclient: found mon.b 2026-02-25T15:07:58.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedbe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedbe7fc640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd0194530 con 0x7fedd015e2a0 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedbe7fc640 10 monclient: _finish_auth 0 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedbe7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedbe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedd7c3c640 5 monclient: authenticate success, global_id 27526 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedd7c3c640 10 monclient: _renew_subs 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedd7c3c640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedd7c3c640 1 -- 10.20.193.90:0/3742749902 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fedd0194810 con 0x7fedd015e2a0 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedd7c3c640 10 client.27526.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedd7c3c640 10 monclient: _renew_subs 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedd7c3c640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedd7c3c640 1 -- 10.20.193.90:0/3742749902 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fedd0195010 con 0x7fedd015e2a0 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedd7c3c640 1 librados: init done 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedd7c3c640 10 client.27526.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedd7c3c640 15 client.27526.objecter _session_command_op_assign -1 1 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedd7c3c640 20 client.27526.objecter _get_session osd=-1 returning homeless 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedd7c3c640 10 client.27526.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedd7c3c640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:58.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.263+0000 7fedd7c3c640 1 -- 10.20.193.90:0/3742749902 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fedd01952a0 con 0x7fedd015e2a0 2026-02-25T15:07:58.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.271+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fedd01952a0 con 0x7fedd015e2a0 2026-02-25T15:07:58.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.271+0000 7fedbe7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:58.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.271+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fedc40039b0 con 0x7fedd015e2a0 2026-02-25T15:07:58.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.271+0000 7fedbe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:58.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.271+0000 7fedbe7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:58.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.271+0000 7fedbe7fc640 10 monclient: dump: 2026-02-25T15:07:58.274 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.274 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.274 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.274 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.274 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.274 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.274 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.274 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.274 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.274 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.271+0000 7fedbffff640 4 set_mon_vals no callback set 2026-02-25T15:07:58.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.272+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fedc4003bc0 con 0x7fedd015e2a0 2026-02-25T15:07:58.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedd59b1640 10 monclient: get_auth_request con 0x7feda007a050 auth_method 0 2026-02-25T15:07:58.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 10 client.27526.objecter ms_handle_connect 0x7feda007a050 2026-02-25T15:07:58.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7fedc40039b0 con 0x7fedd015e2a0 2026-02-25T15:07:58.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 10 client.27526.objecter ms_dispatch2 0x7fedd0000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:58.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 3 client.27526.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:58.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 3 client.27526.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:58.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 10 client.27526.objecter checking command 1 2026-02-25T15:07:58.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 20 client.27526.objecter _get_session s=0x7feda00a4300 osd=6 3 2026-02-25T15:07:58.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 20 client.27526.objecter put_session s=0x7feda00a4300 osd=6 3 2026-02-25T15:07:58.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 15 client.27526.objecter _session_command_op_remove -1 1 2026-02-25T15:07:58.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 20 client.27526.objecter _get_session s=0x7feda00a4300 osd=6 3 2026-02-25T15:07:58.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 20 client.27526.objecter get_session s=0x7feda00a4300 osd=6 3 2026-02-25T15:07:58.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 15 client.27526.objecter _session_command_op_assign 6 1 2026-02-25T15:07:58.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 20 client.27526.objecter put_session s=0x7feda00a4300 osd=6 4 2026-02-25T15:07:58.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 10 client.27526.objecter _send_command 1 2026-02-25T15:07:58.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 --> v1:10.20.193.94:6812/1114158993 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feda00c6610 con 0x7feda00a49b0 2026-02-25T15:07:58.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 20 client.27526.objecter dump_active .. 0 homeless 2026-02-25T15:07:58.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_get_version_reply(handle=1 version=338) ==== 24+0+0 (unknown 1001891302 0 0) 0x7fedc405af00 con 0x7fedd015e2a0 2026-02-25T15:07:58.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.273+0000 7fedbe7fc640 10 monclient: handle_get_version_reply finishing 1 version 338 2026-02-25T15:07:58.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.274+0000 7fedd61b2640 10 monclient: get_auth_request con 0x7feda00a49b0 auth_method 0 2026-02-25T15:07:58.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.275+0000 7fedbe7fc640 10 client.27526.objecter ms_handle_connect 0x7feda00a49b0 2026-02-25T15:07:58.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.285+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== osd.6 v1:10.20.193.94:6812/1114158993 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7feda00c6610 con 0x7feda00a49b0 2026-02-25T15:07:58.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.285+0000 7fedbe7fc640 10 client.27526.objecter ms_dispatch2 0x7fedd0000b90 command_reply(tid 1: 0 ) 2026-02-25T15:07:58.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.285+0000 7fedbe7fc640 10 client.27526.objecter _finish_command 1 = system:0 2026-02-25T15:07:58.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.285+0000 7fedbe7fc640 20 client.27526.objecter put_session s=0x7feda00a4300 osd=6 4 2026-02-25T15:07:58.287 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.285+0000 7fedbe7fc640 15 client.27526.objecter _session_command_op_remove 6 1 2026-02-25T15:07:58.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.292+0000 7fedd7c3c640 10 client.27526.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:07:58.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.292+0000 7fedd7c3c640 15 client.27526.objecter _session_command_op_assign -1 2 2026-02-25T15:07:58.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.292+0000 7fedd7c3c640 20 client.27526.objecter _get_session s=0x7feda00a4300 osd=6 3 2026-02-25T15:07:58.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.292+0000 7fedd7c3c640 20 client.27526.objecter put_session s=0x7feda00a4300 osd=6 3 2026-02-25T15:07:58.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.292+0000 7fedd7c3c640 20 client.27526.objecter _get_session s=0x7feda00a4300 osd=6 3 2026-02-25T15:07:58.294 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.292+0000 7fedd7c3c640 15 client.27526.objecter _session_command_op_remove -1 2 2026-02-25T15:07:58.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.292+0000 7fedd7c3c640 20 client.27526.objecter get_session s=0x7feda00a4300 osd=6 3 2026-02-25T15:07:58.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.292+0000 7fedd7c3c640 15 client.27526.objecter _session_command_op_assign 6 2 2026-02-25T15:07:58.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.292+0000 7fedd7c3c640 20 client.27526.objecter put_session s=0x7feda00a4300 osd=6 4 2026-02-25T15:07:58.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.292+0000 7fedd7c3c640 10 client.27526.objecter _send_command 2 2026-02-25T15:07:58.295 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.292+0000 7fedd7c3c640 1 -- 10.20.193.90:0/3742749902 --> v1:10.20.193.94:6812/1114158993 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fedd0195e10 con 0x7feda00a49b0 2026-02-25T15:07:58.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.299+0000 7fedbe7fc640 1 -- 10.20.193.90:0/3742749902 <== osd.6 v1:10.20.193.94:6812/1114158993 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7fedd0195e10 con 0x7feda00a49b0 2026-02-25T15:07:58.301 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.299+0000 7fedbe7fc640 10 client.27526.objecter ms_dispatch2 0x7fedd0000b90 command_reply(tid 2: 0 ) 2026-02-25T15:07:58.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.299+0000 7fedbe7fc640 10 client.27526.objecter _finish_command 2 = system:0 2026-02-25T15:07:58.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.299+0000 7fedbe7fc640 20 client.27526.objecter put_session s=0x7feda00a4300 osd=6 4 2026-02-25T15:07:58.302 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.299+0000 7fedbe7fc640 15 client.27526.objecter _session_command_op_remove 6 2 2026-02-25T15:07:58.302 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:07:58.302 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-25T15:07:58.302 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:07:58.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 10 librados: watch_flush enter 2026-02-25T15:07:58.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 10 librados: watch_flush exit 2026-02-25T15:07:58.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 10 client.27526.objecter close_session for osd.6 2026-02-25T15:07:58.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 1 -- 10.20.193.90:0/3742749902 >> v1:10.20.193.94:6812/1114158993 conn(0x7feda00a49b0 legacy=0x7feda00c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 20 client.27526.objecter put_session s=0x7feda00a4300 osd=6 1 2026-02-25T15:07:58.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 20 client.27526.objecter shutdown clearing up homeless session... 2026-02-25T15:07:58.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 10 client.27526.objecter successfully canceled tick 2026-02-25T15:07:58.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 1 -- 10.20.193.90:0/3742749902 >> v1:10.20.193.189:6800/3647375982 conn(0x7feda007a050 legacy=0x7feda009a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.304 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 10 monclient: shutdown 2026-02-25T15:07:58.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:58.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 1 -- 10.20.193.90:0/3742749902 >> v1:10.20.193.93:6789/0 conn(0x7fedd015e2a0 legacy=0x7fedd011e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd61b2640 1 -- 10.20.193.90:0/3742749902 reap_dead start 2026-02-25T15:07:58.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 1 -- 10.20.193.90:0/3742749902 shutdown_connections 2026-02-25T15:07:58.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 1 -- 10.20.193.90:0/3742749902 >> 10.20.193.90:0/3742749902 conn(0x7fedd00b2f60 msgr2=0x7fedd01a4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:58.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 1 -- 10.20.193.90:0/3742749902 shutdown_connections 2026-02-25T15:07:58.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 1 -- 10.20.193.90:0/3742749902 wait complete. 2026-02-25T15:07:58.305 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.300+0000 7fedd7c3c640 1 librados: shutdown 2026-02-25T15:07:58.310 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2026-02-25T15:07:58.377 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.376+0000 7fe366af3640 10 monclient: get_monmap_and_config 2026-02-25T15:07:58.377 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.376+0000 7fe366af3640 10 monclient: build_initial_monmap 2026-02-25T15:07:58.378 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.376+0000 7fe366af3640 10 monclient: monmap: 2026-02-25T15:07:58.378 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:58.378 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.378 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:58.376686+0000 2026-02-25T15:07:58.378 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:58.376686+0000 2026-02-25T15:07:58.378 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:58.378 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.378 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:58.378 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:58.378 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:58.379 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.381 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.376+0000 7fe366af3640 1 Processor -- start 2026-02-25T15:07:58.381 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.376+0000 7fe366af3640 1 -- start start 2026-02-25T15:07:58.381 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.376+0000 7fe366af3640 10 monclient: init 2026-02-25T15:07:58.381 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.377+0000 7fe366af3640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:58.381 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.377+0000 7fe366af3640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:07:58.382 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.377+0000 7fe366af3640 10 monclient(hunting): picked mon.noname-a con 0x7fe36015d0c0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:58.382 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.377+0000 7fe366af3640 10 monclient(hunting): picked mon.noname-b con 0x7fe3600b0ca0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:58.382 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.377+0000 7fe366af3640 10 monclient(hunting): picked mon.noname-c con 0x7fe3600b28b0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:58.382 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.377+0000 7fe366af3640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3600b8540 con 0x7fe36015d0c0 2026-02-25T15:07:58.382 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.377+0000 7fe366af3640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3600b5da0 con 0x7fe3600b0ca0 2026-02-25T15:07:58.382 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.377+0000 7fe366af3640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3600b6080 con 0x7fe3600b28b0 2026-02-25T15:07:58.382 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.377+0000 7fe366af3640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:58.382 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.377+0000 7fe366af3640 10 monclient(hunting): authenticate will time out at 3367.977213s 2026-02-25T15:07:58.382 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.377+0000 7fe364868640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fe36015d0c0 0x7fe3600b0590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39960/0 (socket says 10.20.193.90:39960) 2026-02-25T15:07:58.382 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.377+0000 7fe364868640 1 -- 10.20.193.90:0/2888078050 learned_addr learned my addr 10.20.193.90:0/2888078050 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:58.382 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 1 -- 10.20.193.90:0/2888078050 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2697559604 0 0) 0x7fe3600b5da0 con 0x7fe3600b0ca0 2026-02-25T15:07:58.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 10 monclient(hunting): my global_id is 27529 2026-02-25T15:07:58.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 1 -- 10.20.193.90:0/2888078050 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe34c003610 con 0x7fe3600b0ca0 2026-02-25T15:07:58.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 1 -- 10.20.193.90:0/2888078050 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1152867363 0 0) 0x7fe3600b8540 con 0x7fe36015d0c0 2026-02-25T15:07:58.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 10 monclient(hunting): my global_id is 30897 2026-02-25T15:07:58.383 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 1 -- 10.20.193.90:0/2888078050 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3600b5da0 con 0x7fe36015d0c0 2026-02-25T15:07:58.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 1 -- 10.20.193.90:0/2888078050 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3387051167 0 0) 0x7fe3600b6080 con 0x7fe3600b28b0 2026-02-25T15:07:58.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 10 monclient(hunting): my global_id is 28667 2026-02-25T15:07:58.384 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.379+0000 7fe3577fe640 1 -- 10.20.193.90:0/2888078050 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3600b8540 con 0x7fe3600b28b0 2026-02-25T15:07:58.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe3577fe640 1 -- 10.20.193.90:0/2888078050 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3595854750 0 0) 0x7fe34c003610 con 0x7fe3600b0ca0 2026-02-25T15:07:58.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe3577fe640 1 -- 10.20.193.90:0/2888078050 >> v1:10.20.193.94:6789/0 conn(0x7fe3600b28b0 legacy=0x7fe3600b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe3577fe640 1 -- 10.20.193.90:0/2888078050 >> v1:10.20.193.90:6789/0 conn(0x7fe36015d0c0 legacy=0x7fe3600b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe3577fe640 10 monclient: _finish_hunting 0 2026-02-25T15:07:58.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe3577fe640 1 monclient: found mon.noname-b 2026-02-25T15:07:58.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe3577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:58.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe3577fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:07:58.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe3577fe640 1 -- 10.20.193.90:0/2888078050 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe360163e10 con 0x7fe3600b0ca0 2026-02-25T15:07:58.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe3577fe640 10 monclient: _finish_auth 0 2026-02-25T15:07:58.391 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe3577fe640 10 monclient: _check_auth_tickets 2026-02-25T15:07:58.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe3577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:58.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe3577fe640 1 -- 10.20.193.90:0/2888078050 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe348002df0 con 0x7fe3600b0ca0 2026-02-25T15:07:58.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe3577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:58.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe3577fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:58.392 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe3577fe640 10 monclient: dump: 2026-02-25T15:07:58.392 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.392 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.392 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.392 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.392 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.392 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.393 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.393 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.393 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.393 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe366af3640 5 monclient: authenticate success, global_id 27529 2026-02-25T15:07:58.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.380+0000 7fe366af3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:58.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.381+0000 7fe3577fe640 1 -- 10.20.193.90:0/2888078050 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe348002fc0 con 0x7fe3600b0ca0 2026-02-25T15:07:58.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.381+0000 7fe3577fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:58.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.381+0000 7fe3577fe640 1 -- 10.20.193.90:0/2888078050 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe348003400 con 0x7fe3600b0ca0 2026-02-25T15:07:58.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.381+0000 7fe3577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:58.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.381+0000 7fe3577fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:58.393 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.381+0000 7fe3577fe640 10 monclient: dump: 2026-02-25T15:07:58.394 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.394 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.394 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.394 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.394 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.394 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.394 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.394 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.394 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.394 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.381+0000 7fe366af3640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:58.394 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.381+0000 7fe366af3640 4 set_mon_vals no callback set 2026-02-25T15:07:58.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.381+0000 7fe366af3640 10 monclient: shutdown 2026-02-25T15:07:58.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.381+0000 7fe366af3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:58.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.381+0000 7fe366af3640 1 -- 10.20.193.90:0/2888078050 >> v1:10.20.193.93:6789/0 conn(0x7fe3600b0ca0 legacy=0x7fe3600b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.381+0000 7fe366af3640 1 -- 10.20.193.90:0/2888078050 shutdown_connections 2026-02-25T15:07:58.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.381+0000 7fe366af3640 1 -- 10.20.193.90:0/2888078050 >> 10.20.193.90:0/2888078050 conn(0x7fe3600b5290 msgr2=0x7fe3600b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:58.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.381+0000 7fe366af3640 1 -- 10.20.193.90:0/2888078050 shutdown_connections 2026-02-25T15:07:58.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.381+0000 7fe366af3640 1 -- 10.20.193.90:0/2888078050 wait complete. 2026-02-25T15:07:58.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.382+0000 7fe366af3640 10 monclient: build_initial_monmap 2026-02-25T15:07:58.395 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.382+0000 7fe366af3640 10 monclient: monmap: 2026-02-25T15:07:58.395 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:58.395 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:58.395 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:58.395 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:58.396 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:58.396 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:58.396 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:58.396 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:58.396 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:58.396 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.382+0000 7fe366af3640 1 librados: starting msgr at 2026-02-25T15:07:58.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.382+0000 7fe366af3640 1 librados: starting objecter 2026-02-25T15:07:58.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.382+0000 7fe366af3640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:58.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.382+0000 7fe366af3640 1 Processor -- start 2026-02-25T15:07:58.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.382+0000 7fe366af3640 1 -- start start 2026-02-25T15:07:58.396 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.382+0000 7fe366af3640 1 librados: setting wanted keys 2026-02-25T15:07:58.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.382+0000 7fe366af3640 1 librados: calling monclient init 2026-02-25T15:07:58.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.382+0000 7fe366af3640 10 monclient: init 2026-02-25T15:07:58.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe366af3640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:58.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe366af3640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:07:58.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe366af3640 10 monclient(hunting): picked mon.noname-a con 0x7fe3600b0ca0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:58.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe366af3640 10 monclient(hunting): picked mon.noname-c con 0x7fe3600b28b0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:58.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe366af3640 10 monclient(hunting): picked mon.noname-b con 0x7fe36015d0c0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:58.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe366af3640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3601ff200 con 0x7fe3600b0ca0 2026-02-25T15:07:58.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe366af3640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe360138430 con 0x7fe36015d0c0 2026-02-25T15:07:58.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe366af3640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe360138710 con 0x7fe3600b28b0 2026-02-25T15:07:58.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe366af3640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:58.397 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe366af3640 10 monclient(hunting): authenticate will time out at 3367.983114s 2026-02-25T15:07:58.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe364868640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fe3600b0ca0 0x7fe3601feaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:39962/0 (socket says 10.20.193.90:39962) 2026-02-25T15:07:58.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe364868640 1 -- 10.20.193.90:0/344828867 learned_addr learned my addr 10.20.193.90:0/344828867 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:58.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe3557fa640 10 client.?.objecter ms_handle_connect 0x7fe3600b0ca0 2026-02-25T15:07:58.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe3557fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:58.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe3557fa640 10 client.?.objecter ms_handle_connect 0x7fe36015d0c0 2026-02-25T15:07:58.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe3557fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:58.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe3557fa640 10 client.?.objecter ms_handle_connect 0x7fe3600b28b0 2026-02-25T15:07:58.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.383+0000 7fe3557fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:58.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.384+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 859175411 0 0) 0x7fe3601ff200 con 0x7fe3600b0ca0 2026-02-25T15:07:58.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.384+0000 7fe3557fa640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.384+0000 7fe3557fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.398 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.384+0000 7fe3557fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.384+0000 7fe3557fa640 10 monclient(hunting): my global_id is 30900 2026-02-25T15:07:58.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.384+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe330003610 con 0x7fe3600b0ca0 2026-02-25T15:07:58.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.384+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3911585730 0 0) 0x7fe360138430 con 0x7fe36015d0c0 2026-02-25T15:07:58.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.384+0000 7fe3557fa640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.384+0000 7fe3557fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.384+0000 7fe3557fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.384+0000 7fe3557fa640 10 monclient(hunting): my global_id is 27532 2026-02-25T15:07:58.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.384+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3601ff200 con 0x7fe36015d0c0 2026-02-25T15:07:58.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.385+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2811237052 0 0) 0x7fe360138710 con 0x7fe3600b28b0 2026-02-25T15:07:58.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.385+0000 7fe3557fa640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.399 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.385+0000 7fe3557fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.385+0000 7fe3557fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.385+0000 7fe3557fa640 10 monclient(hunting): my global_id is 28670 2026-02-25T15:07:58.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.385+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe360138430 con 0x7fe3600b28b0 2026-02-25T15:07:58.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.386+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3965945988 0 0) 0x7fe330003610 con 0x7fe3600b0ca0 2026-02-25T15:07:58.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.386+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe360138710 con 0x7fe3600b0ca0 2026-02-25T15:07:58.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.386+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe3500030d0 con 0x7fe3600b0ca0 2026-02-25T15:07:58.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.386+0000 7fe3557fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:58.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.386+0000 7fe3557fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:58.400 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.386+0000 7fe3557fa640 10 monclient(hunting): dump: 2026-02-25T15:07:58.400 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.400 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.400 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.401 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.401 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.401 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.401 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.401 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.401 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.401 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.386+0000 7fe3557fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:58.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.386+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3431594400 0 0) 0x7fe3601ff200 con 0x7fe36015d0c0 2026-02-25T15:07:58.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.386+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe330003610 con 0x7fe36015d0c0 2026-02-25T15:07:58.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.386+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe35c003410 con 0x7fe36015d0c0 2026-02-25T15:07:58.401 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.386+0000 7fe3557fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:58.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.386+0000 7fe3557fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:58.402 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.386+0000 7fe3557fa640 10 monclient(hunting): dump: 2026-02-25T15:07:58.402 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.402 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.402 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.402 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.402 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.402 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.402 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.402 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.402 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.402 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3214056354 0 0) 0x7fe360138710 con 0x7fe3600b0ca0 2026-02-25T15:07:58.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 >> v1:10.20.193.94:6789/0 conn(0x7fe3600b28b0 legacy=0x7fe360153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 >> v1:10.20.193.93:6789/0 conn(0x7fe36015d0c0 legacy=0x7fe360152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe3557fa640 10 monclient: _finish_hunting 0 2026-02-25T15:07:58.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe3557fa640 1 monclient: found mon.a 2026-02-25T15:07:58.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe3557fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:58.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe3557fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:58.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3601389f0 con 0x7fe3600b0ca0 2026-02-25T15:07:58.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe3557fa640 10 monclient: _finish_auth 0 2026-02-25T15:07:58.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe3557fa640 10 monclient: _check_auth_tickets 2026-02-25T15:07:58.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe3557fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:58.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe366af3640 5 monclient: authenticate success, global_id 30900 2026-02-25T15:07:58.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe366af3640 10 monclient: _renew_subs 2026-02-25T15:07:58.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe366af3640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:58.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe366af3640 1 -- 10.20.193.90:0/344828867 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe360138cd0 con 0x7fe3600b0ca0 2026-02-25T15:07:58.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe366af3640 10 client.30900.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:58.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe366af3640 10 monclient: _renew_subs 2026-02-25T15:07:58.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe366af3640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:58.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe366af3640 1 -- 10.20.193.90:0/344828867 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe3601566c0 con 0x7fe3600b0ca0 2026-02-25T15:07:58.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe366af3640 1 librados: init done 2026-02-25T15:07:58.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe366af3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:58.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe366af3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:58.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe366af3640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:58.404 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.388+0000 7fe366af3640 1 -- 10.20.193.90:0/344828867 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe360156a00 con 0x7fe3600b0ca0 2026-02-25T15:07:58.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.397+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe3500048d0 con 0x7fe3600b0ca0 2026-02-25T15:07:58.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.397+0000 7fe3557fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:58.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.397+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fe350004c20 con 0x7fe3600b0ca0 2026-02-25T15:07:58.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.397+0000 7fe3557fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:58.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.397+0000 7fe3557fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:58.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.397+0000 7fe3557fa640 10 monclient: dump: 2026-02-25T15:07:58.406 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.406 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.406 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.406 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.406 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.407 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.407 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.407 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.407 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.407 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.397+0000 7fe356ffd640 4 set_mon_vals no callback set 2026-02-25T15:07:58.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.398+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fe35001e100 con 0x7fe3600b0ca0 2026-02-25T15:07:58.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.399+0000 7fe357fff640 10 monclient: get_auth_request con 0x7fe330079f00 auth_method 0 2026-02-25T15:07:58.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.399+0000 7fe3557fa640 10 client.30900.objecter ms_handle_connect 0x7fe330079f00 2026-02-25T15:07:58.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.399+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7fe350095ca0 con 0x7fe3600b0ca0 2026-02-25T15:07:58.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.399+0000 7fe3557fa640 10 client.30900.objecter ms_dispatch2 0x7fe360000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:58.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.399+0000 7fe3557fa640 3 client.30900.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:58.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.399+0000 7fe3557fa640 3 client.30900.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:58.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.399+0000 7fe3557fa640 20 client.30900.objecter dump_active .. 0 homeless 2026-02-25T15:07:58.484 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.483+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fe35005f600 con 0x7fe3600b0ca0 2026-02-25T15:07:58.484 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.483+0000 7fe3557fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:58.484 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.483+0000 7fe3557fa640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:58.592 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.590+0000 7fe366af3640 1 -- 10.20.193.90:0/344828867 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fe3601381e0 con 0x7fe330079f00 2026-02-25T15:07:58.593 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.591+0000 7fe3557fa640 1 -- 10.20.193.90:0/344828867 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (unknown 778718403 0 0) 0x7fe3601381e0 con 0x7fe330079f00 2026-02-25T15:07:58.593 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 6 to deep-scrub 2026-02-25T15:07:58.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.592+0000 7fe366af3640 10 librados: watch_flush enter 2026-02-25T15:07:58.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.592+0000 7fe366af3640 10 librados: watch_flush exit 2026-02-25T15:07:58.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.592+0000 7fe366af3640 20 client.30900.objecter shutdown clearing up homeless session... 2026-02-25T15:07:58.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.592+0000 7fe366af3640 10 client.30900.objecter successfully canceled tick 2026-02-25T15:07:58.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.592+0000 7fe366af3640 1 -- 10.20.193.90:0/344828867 >> v1:10.20.193.189:6800/3647375982 conn(0x7fe330079f00 legacy=0x7fe33009a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.592+0000 7fe366af3640 10 monclient: shutdown 2026-02-25T15:07:58.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.592+0000 7fe366af3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:58.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.592+0000 7fe366af3640 1 -- 10.20.193.90:0/344828867 >> v1:10.20.193.90:6789/0 conn(0x7fe3600b0ca0 legacy=0x7fe3601feaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.592+0000 7fe366af3640 1 -- 10.20.193.90:0/344828867 shutdown_connections 2026-02-25T15:07:58.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.592+0000 7fe366af3640 1 -- 10.20.193.90:0/344828867 >> 10.20.193.90:0/344828867 conn(0x7fe3600b5290 msgr2=0x7fe36015b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:58.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.592+0000 7fe366af3640 1 -- 10.20.193.90:0/344828867 shutdown_connections 2026-02-25T15:07:58.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.592+0000 7fe366af3640 1 -- 10.20.193.90:0/344828867 wait complete. 2026-02-25T15:07:58.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.592+0000 7fe366af3640 1 librados: shutdown 2026-02-25T15:07:58.602 INFO:tasks.ceph:Scrubbing osd.10 2026-02-25T15:07:58.602 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:07:58.673 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.671+0000 7f89a4cd7640 10 monclient: get_monmap_and_config 2026-02-25T15:07:58.673 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.671+0000 7f89a4cd7640 10 monclient: build_initial_monmap 2026-02-25T15:07:58.673 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.671+0000 7f89a4cd7640 10 monclient: monmap: 2026-02-25T15:07:58.673 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:58.673 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.673 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:58.672357+0000 2026-02-25T15:07:58.674 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:58.672357+0000 2026-02-25T15:07:58.674 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:58.674 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.674 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:58.674 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:58.674 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:58.674 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.675 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.672+0000 7f89a4cd7640 1 Processor -- start 2026-02-25T15:07:58.675 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.672+0000 7f89a4cd7640 1 -- start start 2026-02-25T15:07:58.675 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.672+0000 7f89a4cd7640 10 monclient: init 2026-02-25T15:07:58.675 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.672+0000 7f89a4cd7640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:58.675 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.672+0000 7f89a4cd7640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:07:58.675 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.672+0000 7f89a4cd7640 10 monclient(hunting): picked mon.noname-a con 0x7f89a015e2a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:58.675 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.672+0000 7f89a4cd7640 10 monclient(hunting): picked mon.noname-b con 0x7f89a0059e10 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:58.675 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.672+0000 7f89a4cd7640 10 monclient(hunting): picked mon.noname-c con 0x7f89a01a5660 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:58.676 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.672+0000 7f89a4cd7640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89a0162df0 con 0x7f89a015e2a0 2026-02-25T15:07:58.676 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.672+0000 7f89a4cd7640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89a01630d0 con 0x7f89a0059e10 2026-02-25T15:07:58.676 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.672+0000 7f89a4cd7640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89a01633b0 con 0x7f89a01a5660 2026-02-25T15:07:58.676 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.672+0000 7f89a4cd7640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:58.676 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.672+0000 7f89a4cd7640 10 monclient(hunting): authenticate will time out at 3368.272852s 2026-02-25T15:07:58.676 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.673+0000 7f899dd74640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f89a0059e10 0x7f89a015ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45794/0 (socket says 10.20.193.90:45794) 2026-02-25T15:07:58.676 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.673+0000 7f899dd74640 1 -- 10.20.193.90:0/3287119012 learned_addr learned my addr 10.20.193.90:0/3287119012 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:58.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 1 -- 10.20.193.90:0/3287119012 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3570790268 0 0) 0x7f89a0162df0 con 0x7f89a015e2a0 2026-02-25T15:07:58.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 10 monclient(hunting): my global_id is 30903 2026-02-25T15:07:58.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 1 -- 10.20.193.90:0/3287119012 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f898c003610 con 0x7f89a015e2a0 2026-02-25T15:07:58.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 1 -- 10.20.193.90:0/3287119012 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1615463994 0 0) 0x7f89a01633b0 con 0x7f89a01a5660 2026-02-25T15:07:58.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 10 monclient(hunting): my global_id is 28673 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 1 -- 10.20.193.90:0/3287119012 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89a0162df0 con 0x7f89a01a5660 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 1 -- 10.20.193.90:0/3287119012 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3407019627 0 0) 0x7f89a01630d0 con 0x7f89a0059e10 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 10 monclient(hunting): my global_id is 27535 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.674+0000 7f899d573640 1 -- 10.20.193.90:0/3287119012 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89a01633b0 con 0x7f89a0059e10 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f899d573640 1 -- 10.20.193.90:0/3287119012 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3645064476 0 0) 0x7f89a0162df0 con 0x7f89a01a5660 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f899d573640 1 -- 10.20.193.90:0/3287119012 >> v1:10.20.193.93:6789/0 conn(0x7f89a0059e10 legacy=0x7f89a015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f899d573640 1 -- 10.20.193.90:0/3287119012 >> v1:10.20.193.90:6789/0 conn(0x7f89a015e2a0 legacy=0x7f89a005da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f899d573640 10 monclient: _finish_hunting 0 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f899d573640 1 monclient: found mon.noname-c 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f899d573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f899d573640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f899d573640 1 -- 10.20.193.90:0/3287119012 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a0163690 con 0x7f89a01a5660 2026-02-25T15:07:58.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f899d573640 10 monclient: _finish_auth 0 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f899d573640 10 monclient: _check_auth_tickets 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f899d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f899d573640 1 -- 10.20.193.90:0/3287119012 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8990003170 con 0x7f89a01a5660 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f899d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f899d573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f899d573640 10 monclient: dump: 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f89a4cd7640 5 monclient: authenticate success, global_id 28673 2026-02-25T15:07:58.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.676+0000 7f89a4cd7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:58.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.677+0000 7f899d573640 1 -- 10.20.193.90:0/3287119012 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8990003340 con 0x7f89a01a5660 2026-02-25T15:07:58.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.677+0000 7f899d573640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:58.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.677+0000 7f899d573640 1 -- 10.20.193.90:0/3287119012 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8990003780 con 0x7f89a01a5660 2026-02-25T15:07:58.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.677+0000 7f899d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:58.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.677+0000 7f899d573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:58.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.677+0000 7f899d573640 10 monclient: dump: 2026-02-25T15:07:58.684 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.684 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.684 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.684 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.685 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.685 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.685 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.685 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.685 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.685 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.677+0000 7f89a4cd7640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:58.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.677+0000 7f89a4cd7640 4 set_mon_vals no callback set 2026-02-25T15:07:58.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.677+0000 7f89a4cd7640 10 monclient: shutdown 2026-02-25T15:07:58.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.677+0000 7f89a4cd7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:58.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.677+0000 7f89a4cd7640 1 -- 10.20.193.90:0/3287119012 >> v1:10.20.193.94:6789/0 conn(0x7f89a01a5660 legacy=0x7f89a01c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.677+0000 7f89a4cd7640 1 -- 10.20.193.90:0/3287119012 shutdown_connections 2026-02-25T15:07:58.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.677+0000 7f89a4cd7640 1 -- 10.20.193.90:0/3287119012 >> 10.20.193.90:0/3287119012 conn(0x7f89a00b2f60 msgr2=0x7f89a00b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:58.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.677+0000 7f89a4cd7640 1 -- 10.20.193.90:0/3287119012 shutdown_connections 2026-02-25T15:07:58.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.677+0000 7f89a4cd7640 1 -- 10.20.193.90:0/3287119012 wait complete. 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 10 monclient: build_initial_monmap 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 10 monclient: monmap: 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 1 librados: starting msgr at 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 1 librados: starting objecter 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:58.686 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 1 Processor -- start 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 1 -- start start 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 1 librados: setting wanted keys 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 1 librados: calling monclient init 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 10 monclient: init 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 10 monclient(hunting): picked mon.noname-c con 0x7f89a0059e10 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 10 monclient(hunting): picked mon.noname-b con 0x7f89a015e2a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 10 monclient(hunting): picked mon.noname-a con 0x7f89a01a5660 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89a00b24f0 con 0x7f89a01a5660 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89a00a7970 con 0x7f89a015e2a0 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89a00a7c50 con 0x7f89a0059e10 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.678+0000 7f89a4cd7640 10 monclient(hunting): authenticate will time out at 3368.278998s 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.679+0000 7f899dd74640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f89a015e2a0 0x7f89a01941b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45804/0 (socket says 10.20.193.90:45804) 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.679+0000 7f899e575640 1 --1- >> v1:10.20.193.94:6789/0 conn(0x7f89a0059e10 0x7f89a01207d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.94:6789/0 says I am v1:10.20.193.90:54106/0 (socket says 10.20.193.90:54106) 2026-02-25T15:07:58.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.679+0000 7f899dd74640 1 -- 10.20.193.90:0/1860670122 learned_addr learned my addr 10.20.193.90:0/1860670122 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.679+0000 7f8986ffd640 10 client.?.objecter ms_handle_connect 0x7f89a015e2a0 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.679+0000 7f8986ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.679+0000 7f8986ffd640 10 client.?.objecter ms_handle_connect 0x7f89a0059e10 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.679+0000 7f8986ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.679+0000 7f8986ffd640 10 client.?.objecter ms_handle_connect 0x7f89a01a5660 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.679+0000 7f8986ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.680+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3030850765 0 0) 0x7f89a00a7c50 con 0x7f89a0059e10 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.680+0000 7f8986ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.680+0000 7f8986ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.680+0000 7f8986ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.680+0000 7f8986ffd640 10 monclient(hunting): my global_id is 28676 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.680+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f896c003610 con 0x7f89a0059e10 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.680+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1533337456 0 0) 0x7f89a00a7970 con 0x7f89a015e2a0 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.680+0000 7f8986ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.680+0000 7f8986ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.680+0000 7f8986ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.680+0000 7f8986ffd640 10 monclient(hunting): my global_id is 27538 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.680+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89a00a7c50 con 0x7f89a015e2a0 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.681+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2827702848 0 0) 0x7f89a00b24f0 con 0x7f89a01a5660 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.681+0000 7f8986ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.681+0000 7f8986ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.681+0000 7f8986ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.681+0000 7f8986ffd640 10 monclient(hunting): my global_id is 30906 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.681+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89a00a7970 con 0x7f89a01a5660 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.682+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 154956648 0 0) 0x7f89a00a7c50 con 0x7f89a015e2a0 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.682+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f89a00b24f0 con 0x7f89a015e2a0 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.682+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8988002fa0 con 0x7f89a015e2a0 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.682+0000 7f8986ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.682+0000 7f8986ffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.682+0000 7f8986ffd640 10 monclient(hunting): dump: 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.689 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.690 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.690 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.690 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.690 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.690 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.690 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.690 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.682+0000 7f8986ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:58.691 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.683+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 905628727 0 0) 0x7f896c003610 con 0x7f89a0059e10 2026-02-25T15:07:58.692 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.683+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f89a00a7c50 con 0x7f89a0059e10 2026-02-25T15:07:58.692 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.683+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f89940030b0 con 0x7f89a0059e10 2026-02-25T15:07:58.692 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.683+0000 7f8986ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:58.692 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.683+0000 7f8986ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:58.692 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.683+0000 7f8986ffd640 10 monclient(hunting): dump: 2026-02-25T15:07:58.692 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.692 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.692 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.692 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.692 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.692 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.692 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.692 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.692 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.692 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.683+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2640230671 0 0) 0x7f89a00a7970 con 0x7f89a01a5660 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.683+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f896c003610 con 0x7f89a01a5660 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.683+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f8990003690 con 0x7f89a01a5660 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.683+0000 7f8986ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.683+0000 7f8986ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.683+0000 7f8986ffd640 10 monclient(hunting): dump: 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.693 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 173616590 0 0) 0x7f89a00b24f0 con 0x7f89a015e2a0 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 >> v1:10.20.193.94:6789/0 conn(0x7f89a0059e10 legacy=0x7f89a01207d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 >> v1:10.20.193.90:6789/0 conn(0x7f89a01a5660 legacy=0x7f89a012a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f8986ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f8986ffd640 1 monclient: found mon.b 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f8986ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f8986ffd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a00a7f30 con 0x7f89a015e2a0 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f8986ffd640 10 monclient: _finish_auth 0 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f8986ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f8986ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f89a4cd7640 5 monclient: authenticate success, global_id 27538 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f89a4cd7640 10 monclient: _renew_subs 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f89a4cd7640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f89a4cd7640 1 -- 10.20.193.90:0/1860670122 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f89a00a8210 con 0x7f89a015e2a0 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f89a4cd7640 10 client.27538.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:58.694 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f89a4cd7640 10 monclient: _renew_subs 2026-02-25T15:07:58.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f89a4cd7640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:58.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f89a4cd7640 1 -- 10.20.193.90:0/1860670122 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f89a00a87d0 con 0x7f89a015e2a0 2026-02-25T15:07:58.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f89a4cd7640 1 librados: init done 2026-02-25T15:07:58.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f89a4cd7640 10 client.27538.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:58.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f89a4cd7640 15 client.27538.objecter _session_command_op_assign -1 1 2026-02-25T15:07:58.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f89a4cd7640 20 client.27538.objecter _get_session osd=-1 returning homeless 2026-02-25T15:07:58.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f89a4cd7640 10 client.27538.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:58.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f89a4cd7640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:58.695 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.684+0000 7f89a4cd7640 1 -- 10.20.193.90:0/1860670122 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f89a00b27a0 con 0x7f89a015e2a0 2026-02-25T15:07:58.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.692+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f89a00b27a0 con 0x7f89a015e2a0 2026-02-25T15:07:58.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.692+0000 7f8986ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:58.696 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.692+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f89880048d0 con 0x7f89a015e2a0 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.692+0000 7f8986ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.692+0000 7f8986ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.692+0000 7f8986ffd640 10 monclient: dump: 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.692+0000 7f899cd72640 4 set_mon_vals no callback set 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.693+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f8988004ae0 con 0x7f89a015e2a0 2026-02-25T15:07:58.697 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f899e575640 10 monclient: get_auth_request con 0x7f896c079ff0 auth_method 0 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 10 client.27538.objecter ms_handle_connect 0x7f896c079ff0 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7f89880048d0 con 0x7f89a015e2a0 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 10 client.27538.objecter ms_dispatch2 0x7f89a0000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 3 client.27538.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 3 client.27538.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 10 client.27538.objecter checking command 1 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 20 client.27538.objecter _get_session s=0x7f896c0a42a0 osd=10 3 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 20 client.27538.objecter put_session s=0x7f896c0a42a0 osd=10 3 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 15 client.27538.objecter _session_command_op_remove -1 1 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 20 client.27538.objecter _get_session s=0x7f896c0a42a0 osd=10 3 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 20 client.27538.objecter get_session s=0x7f896c0a42a0 osd=10 3 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 15 client.27538.objecter _session_command_op_assign 10 1 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 20 client.27538.objecter put_session s=0x7f896c0a42a0 osd=10 4 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 10 client.27538.objecter _send_command 1 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 --> v1:10.20.193.94:6804/4185080112 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f896c0c65b0 con 0x7f896c0a4950 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 20 client.27538.objecter dump_active .. 0 homeless 2026-02-25T15:07:58.698 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_get_version_reply(handle=1 version=338) ==== 24+0+0 (unknown 1001891302 0 0) 0x7f898805ae70 con 0x7f89a015e2a0 2026-02-25T15:07:58.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.694+0000 7f8986ffd640 10 monclient: handle_get_version_reply finishing 1 version 338 2026-02-25T15:07:58.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.695+0000 7f899ed76640 10 monclient: get_auth_request con 0x7f896c0a4950 auth_method 0 2026-02-25T15:07:58.699 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.696+0000 7f8986ffd640 10 client.27538.objecter ms_handle_connect 0x7f896c0a4950 2026-02-25T15:07:58.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.708+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== osd.10 v1:10.20.193.94:6804/4185080112 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f896c0c65b0 con 0x7f896c0a4950 2026-02-25T15:07:58.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.708+0000 7f8986ffd640 10 client.27538.objecter ms_dispatch2 0x7f89a0000b90 command_reply(tid 1: 0 ) 2026-02-25T15:07:58.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.708+0000 7f8986ffd640 10 client.27538.objecter _finish_command 1 = system:0 2026-02-25T15:07:58.709 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.708+0000 7f8986ffd640 20 client.27538.objecter put_session s=0x7f896c0a42a0 osd=10 4 2026-02-25T15:07:58.710 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.708+0000 7f8986ffd640 15 client.27538.objecter _session_command_op_remove 10 1 2026-02-25T15:07:58.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.715+0000 7f89a4cd7640 10 client.27538.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:07:58.716 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.715+0000 7f89a4cd7640 15 client.27538.objecter _session_command_op_assign -1 2 2026-02-25T15:07:58.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.715+0000 7f89a4cd7640 20 client.27538.objecter _get_session s=0x7f896c0a42a0 osd=10 3 2026-02-25T15:07:58.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.715+0000 7f89a4cd7640 20 client.27538.objecter put_session s=0x7f896c0a42a0 osd=10 3 2026-02-25T15:07:58.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.715+0000 7f89a4cd7640 20 client.27538.objecter _get_session s=0x7f896c0a42a0 osd=10 3 2026-02-25T15:07:58.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.715+0000 7f89a4cd7640 15 client.27538.objecter _session_command_op_remove -1 2 2026-02-25T15:07:58.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.715+0000 7f89a4cd7640 20 client.27538.objecter get_session s=0x7f896c0a42a0 osd=10 3 2026-02-25T15:07:58.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.715+0000 7f89a4cd7640 15 client.27538.objecter _session_command_op_assign 10 2 2026-02-25T15:07:58.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.715+0000 7f89a4cd7640 20 client.27538.objecter put_session s=0x7f896c0a42a0 osd=10 4 2026-02-25T15:07:58.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.715+0000 7f89a4cd7640 10 client.27538.objecter _send_command 2 2026-02-25T15:07:58.717 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.715+0000 7f89a4cd7640 1 -- 10.20.193.90:0/1860670122 --> v1:10.20.193.94:6804/4185080112 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f89a015c670 con 0x7f896c0a4950 2026-02-25T15:07:58.724 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f8986ffd640 1 -- 10.20.193.90:0/1860670122 <== osd.10 v1:10.20.193.94:6804/4185080112 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f89a015c670 con 0x7f896c0a4950 2026-02-25T15:07:58.725 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f8986ffd640 10 client.27538.objecter ms_dispatch2 0x7f89a0000b90 command_reply(tid 2: 0 ) 2026-02-25T15:07:58.725 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f8986ffd640 10 client.27538.objecter _finish_command 2 = system:0 2026-02-25T15:07:58.725 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f8986ffd640 20 client.27538.objecter put_session s=0x7f896c0a42a0 osd=10 4 2026-02-25T15:07:58.725 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f8986ffd640 15 client.27538.objecter _session_command_op_remove 10 2 2026-02-25T15:07:58.725 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:07:58.725 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-25T15:07:58.726 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:07:58.727 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 10 librados: watch_flush enter 2026-02-25T15:07:58.727 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 10 librados: watch_flush exit 2026-02-25T15:07:58.727 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 10 client.27538.objecter close_session for osd.10 2026-02-25T15:07:58.727 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 1 -- 10.20.193.90:0/1860670122 >> v1:10.20.193.94:6804/4185080112 conn(0x7f896c0a4950 legacy=0x7f896c0c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.727 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 20 client.27538.objecter put_session s=0x7f896c0a42a0 osd=10 1 2026-02-25T15:07:58.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 20 client.27538.objecter shutdown clearing up homeless session... 2026-02-25T15:07:58.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 10 client.27538.objecter successfully canceled tick 2026-02-25T15:07:58.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 1 -- 10.20.193.90:0/1860670122 >> v1:10.20.193.189:6800/3647375982 conn(0x7f896c079ff0 legacy=0x7f896c09a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 10 monclient: shutdown 2026-02-25T15:07:58.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:58.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 1 -- 10.20.193.90:0/1860670122 >> v1:10.20.193.93:6789/0 conn(0x7f89a015e2a0 legacy=0x7f89a01941b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f899ed76640 1 -- 10.20.193.90:0/1860670122 reap_dead start 2026-02-25T15:07:58.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 1 -- 10.20.193.90:0/1860670122 shutdown_connections 2026-02-25T15:07:58.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 1 -- 10.20.193.90:0/1860670122 >> 10.20.193.90:0/1860670122 conn(0x7f89a00b2f60 msgr2=0x7f89a015ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:58.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 1 -- 10.20.193.90:0/1860670122 shutdown_connections 2026-02-25T15:07:58.728 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 1 -- 10.20.193.90:0/1860670122 wait complete. 2026-02-25T15:07:58.729 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.723+0000 7f89a4cd7640 1 librados: shutdown 2026-02-25T15:07:58.732 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2026-02-25T15:07:58.802 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.801+0000 7f7c0ec67640 10 monclient: get_monmap_and_config 2026-02-25T15:07:58.803 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.801+0000 7f7c0ec67640 10 monclient: build_initial_monmap 2026-02-25T15:07:58.803 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.801+0000 7f7c0ec67640 10 monclient: monmap: 2026-02-25T15:07:58.803 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:58.803 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.803 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:58.801796+0000 2026-02-25T15:07:58.803 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:58.801796+0000 2026-02-25T15:07:58.803 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:58.803 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.803 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:58.803 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:58.803 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:58.804 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.806 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.801+0000 7f7c0ec67640 1 Processor -- start 2026-02-25T15:07:58.806 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.801+0000 7f7c0ec67640 1 -- start start 2026-02-25T15:07:58.806 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.801+0000 7f7c0ec67640 10 monclient: init 2026-02-25T15:07:58.806 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.802+0000 7f7c0ec67640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:58.806 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.802+0000 7f7c0ec67640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:07:58.806 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.802+0000 7f7c0ec67640 10 monclient(hunting): picked mon.noname-b con 0x7f7c08157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:58.806 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.802+0000 7f7c0ec67640 10 monclient(hunting): picked mon.noname-a con 0x7f7c08157b50 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:58.807 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.802+0000 7f7c0ec67640 10 monclient(hunting): picked mon.noname-c con 0x7f7c08164d00 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:58.807 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.802+0000 7f7c0ec67640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c0805de70 con 0x7f7c08157b50 2026-02-25T15:07:58.807 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.802+0000 7f7c0ec67640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c08055b20 con 0x7f7c08157780 2026-02-25T15:07:58.807 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.802+0000 7f7c0ec67640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c08055e00 con 0x7f7c08164d00 2026-02-25T15:07:58.807 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.802+0000 7f7c0ec67640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:58.807 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.802+0000 7f7c0ec67640 10 monclient(hunting): authenticate will time out at 3368.402229s 2026-02-25T15:07:58.807 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.802+0000 7f7c0c9dc640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f7c08157780 0x7f7c08056ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45806/0 (socket says 10.20.193.90:45806) 2026-02-25T15:07:58.807 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.802+0000 7f7c0c9dc640 1 -- 10.20.193.90:0/835854584 learned_addr learned my addr 10.20.193.90:0/835854584 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:58.807 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.803+0000 7f7bff7fe640 1 -- 10.20.193.90:0/835854584 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 501528222 0 0) 0x7f7c0805de70 con 0x7f7c08157b50 2026-02-25T15:07:58.807 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.803+0000 7f7bff7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.807 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.803+0000 7f7bff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.807 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.803+0000 7f7bff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.808 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.803+0000 7f7bff7fe640 10 monclient(hunting): my global_id is 30909 2026-02-25T15:07:58.808 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.803+0000 7f7bff7fe640 1 -- 10.20.193.90:0/835854584 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bf4003610 con 0x7f7c08157b50 2026-02-25T15:07:58.808 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.804+0000 7f7bff7fe640 1 -- 10.20.193.90:0/835854584 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2430720364 0 0) 0x7f7c08055b20 con 0x7f7c08157780 2026-02-25T15:07:58.808 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.804+0000 7f7bff7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.808 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.804+0000 7f7bff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.808 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.804+0000 7f7bff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.808 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.804+0000 7f7bff7fe640 10 monclient(hunting): my global_id is 27541 2026-02-25T15:07:58.808 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.804+0000 7f7bff7fe640 1 -- 10.20.193.90:0/835854584 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c0805de70 con 0x7f7c08157780 2026-02-25T15:07:58.808 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.804+0000 7f7bff7fe640 1 -- 10.20.193.90:0/835854584 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2013288549 0 0) 0x7f7c08055e00 con 0x7f7c08164d00 2026-02-25T15:07:58.808 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.804+0000 7f7bff7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.808 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.804+0000 7f7bff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.808 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.804+0000 7f7bff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.809 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.804+0000 7f7bff7fe640 10 monclient(hunting): my global_id is 28679 2026-02-25T15:07:58.809 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.804+0000 7f7bff7fe640 1 -- 10.20.193.90:0/835854584 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c08055b20 con 0x7f7c08164d00 2026-02-25T15:07:58.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7bff7fe640 1 -- 10.20.193.90:0/835854584 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2591799479 0 0) 0x7f7bf4003610 con 0x7f7c08157b50 2026-02-25T15:07:58.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7bff7fe640 1 -- 10.20.193.90:0/835854584 >> v1:10.20.193.94:6789/0 conn(0x7f7c08164d00 legacy=0x7f7c081c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.814 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7bff7fe640 1 -- 10.20.193.90:0/835854584 >> v1:10.20.193.93:6789/0 conn(0x7f7c08157780 legacy=0x7f7c08056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7bff7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:07:58.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7bff7fe640 1 monclient: found mon.noname-a 2026-02-25T15:07:58.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7bff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:58.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7bff7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:07:58.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7bff7fe640 1 -- 10.20.193.90:0/835854584 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c080560e0 con 0x7f7c08157b50 2026-02-25T15:07:58.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7bff7fe640 10 monclient: _finish_auth 0 2026-02-25T15:07:58.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7bff7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:07:58.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7bff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:58.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7bff7fe640 1 -- 10.20.193.90:0/835854584 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7bf0002df0 con 0x7f7c08157b50 2026-02-25T15:07:58.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7bff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:58.815 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7bff7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:58.816 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7bff7fe640 10 monclient: dump: 2026-02-25T15:07:58.816 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.816 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.816 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.816 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.816 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.816 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.816 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.816 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.816 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.816 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7c0ec67640 5 monclient: authenticate success, global_id 30909 2026-02-25T15:07:58.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.805+0000 7f7c0ec67640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:58.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.806+0000 7f7bff7fe640 1 -- 10.20.193.90:0/835854584 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7bf0002fc0 con 0x7f7c08157b50 2026-02-25T15:07:58.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.806+0000 7f7bff7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:58.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.806+0000 7f7bff7fe640 1 -- 10.20.193.90:0/835854584 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7bf0003400 con 0x7f7c08157b50 2026-02-25T15:07:58.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.806+0000 7f7bff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:58.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.806+0000 7f7bff7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:58.817 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.806+0000 7f7bff7fe640 10 monclient: dump: 2026-02-25T15:07:58.817 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.817 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.817 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.817 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.818 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.818 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.818 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.818 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.818 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.818 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.806+0000 7f7c0ec67640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:58.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.806+0000 7f7c0ec67640 4 set_mon_vals no callback set 2026-02-25T15:07:58.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.806+0000 7f7c0ec67640 10 monclient: shutdown 2026-02-25T15:07:58.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.806+0000 7f7c0ec67640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:58.818 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.806+0000 7f7c0ec67640 1 -- 10.20.193.90:0/835854584 >> v1:10.20.193.90:6789/0 conn(0x7f7c08157b50 legacy=0x7f7c081645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.806+0000 7f7c0ec67640 1 -- 10.20.193.90:0/835854584 shutdown_connections 2026-02-25T15:07:58.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.806+0000 7f7c0ec67640 1 -- 10.20.193.90:0/835854584 >> 10.20.193.90:0/835854584 conn(0x7f7c080b2ef0 msgr2=0x7f7c080b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:58.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.806+0000 7f7c0ec67640 1 -- 10.20.193.90:0/835854584 shutdown_connections 2026-02-25T15:07:58.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.806+0000 7f7c0ec67640 1 -- 10.20.193.90:0/835854584 wait complete. 2026-02-25T15:07:58.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.807+0000 7f7c0ec67640 10 monclient: build_initial_monmap 2026-02-25T15:07:58.819 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.807+0000 7f7c0ec67640 10 monclient: monmap: 2026-02-25T15:07:58.819 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:58.819 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:58.819 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:58.819 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:58.819 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:58.819 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:58.819 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:58.820 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:58.820 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:58.820 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.807+0000 7f7c0ec67640 1 librados: starting msgr at 2026-02-25T15:07:58.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.807+0000 7f7c0ec67640 1 librados: starting objecter 2026-02-25T15:07:58.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.807+0000 7f7c0ec67640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:58.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.807+0000 7f7c0ec67640 1 Processor -- start 2026-02-25T15:07:58.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.807+0000 7f7c0ec67640 1 -- start start 2026-02-25T15:07:58.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.807+0000 7f7c0ec67640 1 librados: setting wanted keys 2026-02-25T15:07:58.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.807+0000 7f7c0ec67640 1 librados: calling monclient init 2026-02-25T15:07:58.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.807+0000 7f7c0ec67640 10 monclient: init 2026-02-25T15:07:58.820 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7c0ec67640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:58.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7c0ec67640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:07:58.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7c0ec67640 10 monclient(hunting): picked mon.noname-a con 0x7f7c08157780 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:58.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7c0ec67640 10 monclient(hunting): picked mon.noname-c con 0x7f7c08157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:58.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7c0ec67640 10 monclient(hunting): picked mon.noname-b con 0x7f7c08164d00 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:58.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7c0ec67640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c080affd0 con 0x7f7c08157780 2026-02-25T15:07:58.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7c0ec67640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c08136ac0 con 0x7f7c08164d00 2026-02-25T15:07:58.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7c0ec67640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c08136da0 con 0x7f7c08157b50 2026-02-25T15:07:58.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7c0ec67640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:58.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7c0ec67640 10 monclient(hunting): authenticate will time out at 3368.408097s 2026-02-25T15:07:58.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7c0c9dc640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f7c08157780 0x7f7c080af8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:40000/0 (socket says 10.20.193.90:40000) 2026-02-25T15:07:58.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7c0c9dc640 1 -- 10.20.193.90:0/3212442584 learned_addr learned my addr 10.20.193.90:0/3212442584 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:58.821 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7bfd7fa640 10 client.?.objecter ms_handle_connect 0x7f7c08157780 2026-02-25T15:07:58.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7bfd7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:58.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7bfd7fa640 10 client.?.objecter ms_handle_connect 0x7f7c08164d00 2026-02-25T15:07:58.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7bfd7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:58.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7bfd7fa640 10 client.?.objecter ms_handle_connect 0x7f7c08157b50 2026-02-25T15:07:58.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.808+0000 7f7bfd7fa640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:58.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1090323220 0 0) 0x7f7c080affd0 con 0x7f7c08157780 2026-02-25T15:07:58.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 10 monclient(hunting): my global_id is 30912 2026-02-25T15:07:58.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bd8003610 con 0x7f7c08157780 2026-02-25T15:07:58.822 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1828135350 0 0) 0x7f7c08136da0 con 0x7f7c08157b50 2026-02-25T15:07:58.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 10 monclient(hunting): my global_id is 28682 2026-02-25T15:07:58.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c080affd0 con 0x7f7c08157b50 2026-02-25T15:07:58.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1494630169 0 0) 0x7f7c08136ac0 con 0x7f7c08164d00 2026-02-25T15:07:58.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 10 monclient(hunting): _negotiate 2026-02-25T15:07:58.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:58.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:58.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 10 monclient(hunting): my global_id is 27544 2026-02-25T15:07:58.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.809+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c08136da0 con 0x7f7c08164d00 2026-02-25T15:07:58.823 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.811+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2814077094 0 0) 0x7f7bd8003610 con 0x7f7c08157780 2026-02-25T15:07:58.824 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.811+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7c08136ac0 con 0x7f7c08157780 2026-02-25T15:07:58.824 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.811+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7bf8003170 con 0x7f7c08157780 2026-02-25T15:07:58.824 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.811+0000 7f7bfd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:58.824 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.811+0000 7f7bfd7fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:58.824 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.811+0000 7f7bfd7fa640 10 monclient(hunting): dump: 2026-02-25T15:07:58.824 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.824 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.824 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.824 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.824 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.824 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.824 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.825 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.825 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.825 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.811+0000 7f7bfd7fa640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:58.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.811+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 154439750 0 0) 0x7f7c08136da0 con 0x7f7c08164d00 2026-02-25T15:07:58.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.811+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7bd8003610 con 0x7f7c08164d00 2026-02-25T15:07:58.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.811+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7c04003330 con 0x7f7c08164d00 2026-02-25T15:07:58.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.811+0000 7f7bfd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:58.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.811+0000 7f7bfd7fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:58.825 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.811+0000 7f7bfd7fa640 10 monclient(hunting): dump: 2026-02-25T15:07:58.825 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.825 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.826 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.826 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.826 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.826 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.826 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.826 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.826 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.826 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.812+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1027130542 0 0) 0x7f7c080affd0 con 0x7f7c08157b50 2026-02-25T15:07:58.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.812+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7c08136da0 con 0x7f7c08157b50 2026-02-25T15:07:58.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.812+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7bf0003310 con 0x7f7c08157b50 2026-02-25T15:07:58.826 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.812+0000 7f7bfd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:58.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.812+0000 7f7bfd7fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:58.827 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.812+0000 7f7bfd7fa640 10 monclient(hunting): dump: 2026-02-25T15:07:58.827 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.827 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.827 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.827 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.827 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.827 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.827 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.827 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.827 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.827 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2748095769 0 0) 0x7f7bd8003610 con 0x7f7c08164d00 2026-02-25T15:07:58.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 >> v1:10.20.193.94:6789/0 conn(0x7f7c08157b50 legacy=0x7f7c08137b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 >> v1:10.20.193.90:6789/0 conn(0x7f7c08157780 legacy=0x7f7c080af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:58.830 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7bfd7fa640 10 monclient: _finish_hunting 0 2026-02-25T15:07:58.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7bfd7fa640 1 monclient: found mon.b 2026-02-25T15:07:58.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7bfd7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:58.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7bfd7fa640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:58.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c08137080 con 0x7f7c08164d00 2026-02-25T15:07:58.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7bfd7fa640 10 monclient: _finish_auth 0 2026-02-25T15:07:58.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7bfd7fa640 10 monclient: _check_auth_tickets 2026-02-25T15:07:58.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7bfd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:58.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7c0ec67640 5 monclient: authenticate success, global_id 27544 2026-02-25T15:07:58.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7c0ec67640 10 monclient: _renew_subs 2026-02-25T15:07:58.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7c0ec67640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:58.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7c0ec67640 1 -- 10.20.193.90:0/3212442584 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7c08137360 con 0x7f7c08164d00 2026-02-25T15:07:58.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7c0ec67640 10 client.27544.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:58.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7c0ec67640 10 monclient: _renew_subs 2026-02-25T15:07:58.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7c0ec67640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:58.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7c0ec67640 1 -- 10.20.193.90:0/3212442584 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7c08139f00 con 0x7f7c08164d00 2026-02-25T15:07:58.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.813+0000 7f7c0ec67640 1 librados: init done 2026-02-25T15:07:58.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.814+0000 7f7c0ec67640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:58.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.814+0000 7f7c0ec67640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:58.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.814+0000 7f7c0ec67640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:07:58.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.814+0000 7f7c0ec67640 1 -- 10.20.193.90:0/3212442584 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c080a3070 con 0x7f7c08164d00 2026-02-25T15:07:58.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.822+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7c04003860 con 0x7f7c08164d00 2026-02-25T15:07:58.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.822+0000 7f7bfd7fa640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:58.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.822+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7c04003c20 con 0x7f7c08164d00 2026-02-25T15:07:58.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.822+0000 7f7bfd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:58.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.822+0000 7f7bfd7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:58.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.822+0000 7f7bfd7fa640 10 monclient: dump: 2026-02-25T15:07:58.833 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:58.833 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:58.833 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.833 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:58.833 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:58.833 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:58.833 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:58.833 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:58.833 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:58.833 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:58.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.822+0000 7f7bfeffd640 4 set_mon_vals no callback set 2026-02-25T15:07:58.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.823+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f7c0401d100 con 0x7f7c08164d00 2026-02-25T15:07:58.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.824+0000 7f7c0c9dc640 10 monclient: get_auth_request con 0x7f7bd807a180 auth_method 0 2026-02-25T15:07:58.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.824+0000 7f7bfd7fa640 10 client.27544.objecter ms_handle_connect 0x7f7bd807a180 2026-02-25T15:07:58.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.825+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7f7c04094a90 con 0x7f7c08164d00 2026-02-25T15:07:58.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.825+0000 7f7bfd7fa640 10 client.27544.objecter ms_dispatch2 0x7f7c08000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:58.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.825+0000 7f7bfd7fa640 3 client.27544.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:58.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.825+0000 7f7bfd7fa640 3 client.27544.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:58.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.825+0000 7f7bfd7fa640 20 client.27544.objecter dump_active .. 0 homeless 2026-02-25T15:07:58.913 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.912+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f7c0405e3f0 con 0x7f7c08164d00 2026-02-25T15:07:58.914 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.912+0000 7f7bfd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:58.914 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:58.912+0000 7f7bfd7fa640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:59.021 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.019+0000 7f7c0ec67640 1 -- 10.20.193.90:0/3212442584 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "10", "target": ["mon-mgr", ""]}) -- 0x7f7c080b4ff0 con 0x7f7bd807a180 2026-02-25T15:07:59.021 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.019+0000 7f7bfd7fa640 1 -- 10.20.193.90:0/3212442584 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 10 to deep-scrub) ==== 42+0+0 (unknown 3896554461 0 0) 0x7f7c080b4ff0 con 0x7f7bd807a180 2026-02-25T15:07:59.021 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 10 to deep-scrub 2026-02-25T15:07:59.023 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.020+0000 7f7c0ec67640 10 librados: watch_flush enter 2026-02-25T15:07:59.023 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.020+0000 7f7c0ec67640 10 librados: watch_flush exit 2026-02-25T15:07:59.023 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.020+0000 7f7c0ec67640 20 client.27544.objecter shutdown clearing up homeless session... 2026-02-25T15:07:59.023 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.020+0000 7f7c0ec67640 10 client.27544.objecter successfully canceled tick 2026-02-25T15:07:59.023 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.020+0000 7f7c0ec67640 1 -- 10.20.193.90:0/3212442584 >> v1:10.20.193.189:6800/3647375982 conn(0x7f7bd807a180 legacy=0x7f7bd809a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.023 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.020+0000 7f7c0ec67640 10 monclient: shutdown 2026-02-25T15:07:59.023 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.020+0000 7f7c0ec67640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:59.023 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.020+0000 7f7c0ec67640 1 -- 10.20.193.90:0/3212442584 >> v1:10.20.193.93:6789/0 conn(0x7f7c08164d00 legacy=0x7f7c0805c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.024 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.020+0000 7f7c0ec67640 1 -- 10.20.193.90:0/3212442584 shutdown_connections 2026-02-25T15:07:59.024 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.020+0000 7f7c0ec67640 1 -- 10.20.193.90:0/3212442584 >> 10.20.193.90:0/3212442584 conn(0x7f7c080b2ef0 msgr2=0x7f7c0813bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:59.024 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.020+0000 7f7c0ec67640 1 -- 10.20.193.90:0/3212442584 shutdown_connections 2026-02-25T15:07:59.024 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.021+0000 7f7c0ec67640 1 -- 10.20.193.90:0/3212442584 wait complete. 2026-02-25T15:07:59.024 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.021+0000 7f7c0ec67640 1 librados: shutdown 2026-02-25T15:07:59.031 INFO:tasks.ceph:Scrubbing osd.14 2026-02-25T15:07:59.032 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:07:59.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.100+0000 7f787c016640 10 monclient: get_monmap_and_config 2026-02-25T15:07:59.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.100+0000 7f787c016640 10 monclient: build_initial_monmap 2026-02-25T15:07:59.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.100+0000 7f787c016640 10 monclient: monmap: 2026-02-25T15:07:59.106 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:59.106 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.106 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:59.101138+0000 2026-02-25T15:07:59.106 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:59.101138+0000 2026-02-25T15:07:59.106 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:59.106 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.106 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:59.106 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:59.106 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:59.106 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.100+0000 7f787c016640 1 Processor -- start 2026-02-25T15:07:59.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.100+0000 7f787c016640 1 -- start start 2026-02-25T15:07:59.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.100+0000 7f787c016640 10 monclient: init 2026-02-25T15:07:59.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.101+0000 7f787c016640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:59.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.101+0000 7f787c016640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:07:59.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.101+0000 7f787c016640 10 monclient(hunting): picked mon.noname-a con 0x7f787415e2a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:59.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.101+0000 7f787c016640 10 monclient(hunting): picked mon.noname-b con 0x7f7874059e10 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:59.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.101+0000 7f787c016640 10 monclient(hunting): picked mon.noname-c con 0x7f78741a5660 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:59.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.101+0000 7f787c016640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7874162df0 con 0x7f787415e2a0 2026-02-25T15:07:59.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.101+0000 7f787c016640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78741630d0 con 0x7f7874059e10 2026-02-25T15:07:59.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.101+0000 7f787c016640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78741633b0 con 0x7f78741a5660 2026-02-25T15:07:59.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.101+0000 7f787c016640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:59.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.101+0000 7f787c016640 10 monclient(hunting): authenticate will time out at 3368.701499s 2026-02-25T15:07:59.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.101+0000 7f7879d8b640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f787415e2a0 0x7f787405da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:40010/0 (socket says 10.20.193.90:40010) 2026-02-25T15:07:59.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.101+0000 7f7879d8b640 1 -- 10.20.193.90:0/1827134825 learned_addr learned my addr 10.20.193.90:0/1827134825 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:59.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.102+0000 7f7878d89640 1 -- 10.20.193.90:0/1827134825 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1053524006 0 0) 0x7f7874162df0 con 0x7f787415e2a0 2026-02-25T15:07:59.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.102+0000 7f7878d89640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.102+0000 7f7878d89640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.102+0000 7f7878d89640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.102+0000 7f7878d89640 10 monclient(hunting): my global_id is 30915 2026-02-25T15:07:59.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.103+0000 7f7878d89640 1 -- 10.20.193.90:0/1827134825 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f785c003610 con 0x7f787415e2a0 2026-02-25T15:07:59.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.103+0000 7f7878d89640 1 -- 10.20.193.90:0/1827134825 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1942985985 0 0) 0x7f78741633b0 con 0x7f78741a5660 2026-02-25T15:07:59.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.103+0000 7f7878d89640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.103+0000 7f7878d89640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.103+0000 7f7878d89640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.103+0000 7f7878d89640 10 monclient(hunting): my global_id is 28685 2026-02-25T15:07:59.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.103+0000 7f7878d89640 1 -- 10.20.193.90:0/1827134825 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7874162df0 con 0x7f78741a5660 2026-02-25T15:07:59.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.103+0000 7f7878d89640 1 -- 10.20.193.90:0/1827134825 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1242770295 0 0) 0x7f78741630d0 con 0x7f7874059e10 2026-02-25T15:07:59.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.103+0000 7f7878d89640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.103+0000 7f7878d89640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.103+0000 7f7878d89640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.103+0000 7f7878d89640 10 monclient(hunting): my global_id is 27547 2026-02-25T15:07:59.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.103+0000 7f7878d89640 1 -- 10.20.193.90:0/1827134825 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78741633b0 con 0x7f7874059e10 2026-02-25T15:07:59.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f7878d89640 1 -- 10.20.193.90:0/1827134825 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 265325926 0 0) 0x7f785c003610 con 0x7f787415e2a0 2026-02-25T15:07:59.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f7878d89640 1 -- 10.20.193.90:0/1827134825 >> v1:10.20.193.94:6789/0 conn(0x7f78741a5660 legacy=0x7f78741c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f7878d89640 1 -- 10.20.193.90:0/1827134825 >> v1:10.20.193.93:6789/0 conn(0x7f7874059e10 legacy=0x7f787415ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f7878d89640 10 monclient: _finish_hunting 0 2026-02-25T15:07:59.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f7878d89640 1 monclient: found mon.noname-a 2026-02-25T15:07:59.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f7878d89640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:59.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f7878d89640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f7878d89640 1 -- 10.20.193.90:0/1827134825 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7874163690 con 0x7f787415e2a0 2026-02-25T15:07:59.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f7878d89640 10 monclient: _finish_auth 0 2026-02-25T15:07:59.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f7878d89640 10 monclient: _check_auth_tickets 2026-02-25T15:07:59.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f7878d89640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:59.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f7878d89640 1 -- 10.20.193.90:0/1827134825 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7864002df0 con 0x7f787415e2a0 2026-02-25T15:07:59.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f787c016640 5 monclient: authenticate success, global_id 30915 2026-02-25T15:07:59.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f787c016640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:59.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f7878d89640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:59.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f7878d89640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:59.118 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f7878d89640 10 monclient: dump: 2026-02-25T15:07:59.118 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.118 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.118 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.118 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.119 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.119 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.119 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.119 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.119 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.119 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.104+0000 7f787c016640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:59.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.105+0000 7f7878d89640 1 -- 10.20.193.90:0/1827134825 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7864002fc0 con 0x7f787415e2a0 2026-02-25T15:07:59.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.105+0000 7f7878d89640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:59.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.105+0000 7f7878d89640 1 -- 10.20.193.90:0/1827134825 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7864003400 con 0x7f787415e2a0 2026-02-25T15:07:59.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.105+0000 7f787c016640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:59.119 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.105+0000 7f787c016640 4 set_mon_vals no callback set 2026-02-25T15:07:59.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.105+0000 7f7878d89640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:59.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.105+0000 7f7878d89640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:59.120 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.105+0000 7f7878d89640 10 monclient: dump: 2026-02-25T15:07:59.120 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.120 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.120 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.120 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.120 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.120 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.120 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.120 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.120 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.121 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.105+0000 7f787c016640 10 monclient: shutdown 2026-02-25T15:07:59.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.105+0000 7f787c016640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:59.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.105+0000 7f787c016640 1 -- 10.20.193.90:0/1827134825 >> v1:10.20.193.90:6789/0 conn(0x7f787415e2a0 legacy=0x7f787405da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.105+0000 7f787c016640 1 -- 10.20.193.90:0/1827134825 shutdown_connections 2026-02-25T15:07:59.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.105+0000 7f787c016640 1 -- 10.20.193.90:0/1827134825 >> 10.20.193.90:0/1827134825 conn(0x7f78740b2f60 msgr2=0x7f78740b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:59.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.105+0000 7f787c016640 1 -- 10.20.193.90:0/1827134825 shutdown_connections 2026-02-25T15:07:59.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.105+0000 7f787c016640 1 -- 10.20.193.90:0/1827134825 wait complete. 2026-02-25T15:07:59.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.106+0000 7f787c016640 10 monclient: build_initial_monmap 2026-02-25T15:07:59.121 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.106+0000 7f787c016640 10 monclient: monmap: 2026-02-25T15:07:59.121 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:59.121 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:59.121 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:59.122 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:59.122 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:59.122 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:59.122 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:59.122 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:59.122 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:59.122 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.106+0000 7f787c016640 1 librados: starting msgr at 2026-02-25T15:07:59.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.106+0000 7f787c016640 1 librados: starting objecter 2026-02-25T15:07:59.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.106+0000 7f787c016640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:59.122 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.106+0000 7f787c016640 1 Processor -- start 2026-02-25T15:07:59.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.106+0000 7f787c016640 1 -- start start 2026-02-25T15:07:59.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.106+0000 7f787c016640 1 librados: setting wanted keys 2026-02-25T15:07:59.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.106+0000 7f787c016640 1 librados: calling monclient init 2026-02-25T15:07:59.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.106+0000 7f787c016640 10 monclient: init 2026-02-25T15:07:59.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f787c016640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:59.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f787c016640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-25T15:07:59.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f787c016640 10 monclient(hunting): picked mon.noname-b con 0x7f7874059e10 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:59.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f787c016640 10 monclient(hunting): picked mon.noname-a con 0x7f787415e2a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:59.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f787c016640 10 monclient(hunting): picked mon.noname-c con 0x7f78741a5660 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:59.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f787c016640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7874195040 con 0x7f787415e2a0 2026-02-25T15:07:59.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f787c016640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7874195320 con 0x7f7874059e10 2026-02-25T15:07:59.123 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f787c016640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7874195600 con 0x7f78741a5660 2026-02-25T15:07:59.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f787c016640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:59.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f787c016640 10 monclient(hunting): authenticate will time out at 3368.707056s 2026-02-25T15:07:59.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f7879d8b640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f7874059e10 0x7f7874194870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45836/0 (socket says 10.20.193.90:45836) 2026-02-25T15:07:59.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f7879d8b640 1 -- 10.20.193.90:0/3969952941 learned_addr learned my addr 10.20.193.90:0/3969952941 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:59.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f78627fc640 10 client.?.objecter ms_handle_connect 0x7f7874059e10 2026-02-25T15:07:59.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f78627fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:59.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f78627fc640 10 client.?.objecter ms_handle_connect 0x7f78741a5660 2026-02-25T15:07:59.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f78627fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:59.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f78627fc640 10 client.?.objecter ms_handle_connect 0x7f787415e2a0 2026-02-25T15:07:59.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.107+0000 7f78627fc640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:59.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3783080848 0 0) 0x7f7874195040 con 0x7f787415e2a0 2026-02-25T15:07:59.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.124 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 10 monclient(hunting): my global_id is 30918 2026-02-25T15:07:59.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7844003610 con 0x7f787415e2a0 2026-02-25T15:07:59.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1308975958 0 0) 0x7f7874195600 con 0x7f78741a5660 2026-02-25T15:07:59.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 10 monclient(hunting): my global_id is 28688 2026-02-25T15:07:59.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7874195040 con 0x7f78741a5660 2026-02-25T15:07:59.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 592072475 0 0) 0x7f7874195320 con 0x7f7874059e10 2026-02-25T15:07:59.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.125 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 10 monclient(hunting): my global_id is 27550 2026-02-25T15:07:59.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.108+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7874195600 con 0x7f7874059e10 2026-02-25T15:07:59.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.110+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2315610289 0 0) 0x7f7844003610 con 0x7f787415e2a0 2026-02-25T15:07:59.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.110+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7874195320 con 0x7f787415e2a0 2026-02-25T15:07:59.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.110+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7868002fb0 con 0x7f787415e2a0 2026-02-25T15:07:59.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.110+0000 7f78627fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:59.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.110+0000 7f78627fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:59.126 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.110+0000 7f78627fc640 10 monclient(hunting): dump: 2026-02-25T15:07:59.126 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.126 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.126 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.127 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.127 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.127 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.127 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.127 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.127 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.127 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.110+0000 7f78627fc640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:59.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.111+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1549773861 0 0) 0x7f7874195040 con 0x7f78741a5660 2026-02-25T15:07:59.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.111+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7844003610 con 0x7f78741a5660 2026-02-25T15:07:59.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.111+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7870003430 con 0x7f78741a5660 2026-02-25T15:07:59.127 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.111+0000 7f78627fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:59.128 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.111+0000 7f78627fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:59.128 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.111+0000 7f78627fc640 10 monclient(hunting): dump: 2026-02-25T15:07:59.128 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.128 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.128 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.128 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.128 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.128 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.128 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.128 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.128 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.128 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.111+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 831776387 0 0) 0x7f7874195600 con 0x7f7874059e10 2026-02-25T15:07:59.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.111+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7874195040 con 0x7f7874059e10 2026-02-25T15:07:59.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.111+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7864003310 con 0x7f7874059e10 2026-02-25T15:07:59.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.111+0000 7f78627fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:59.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.111+0000 7f78627fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:59.129 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.111+0000 7f78627fc640 10 monclient(hunting): dump: 2026-02-25T15:07:59.129 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.129 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.129 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.129 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.129 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.129 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.129 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.130 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.130 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.130 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1179289216 0 0) 0x7f7874195320 con 0x7f787415e2a0 2026-02-25T15:07:59.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 >> v1:10.20.193.94:6789/0 conn(0x7f78741a5660 legacy=0x7f78740b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 >> v1:10.20.193.93:6789/0 conn(0x7f7874059e10 legacy=0x7f7874194870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f78627fc640 10 monclient: _finish_hunting 0 2026-02-25T15:07:59.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f78627fc640 1 monclient: found mon.a 2026-02-25T15:07:59.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f78627fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:59.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f78627fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7874190670 con 0x7f787415e2a0 2026-02-25T15:07:59.130 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f78627fc640 10 monclient: _finish_auth 0 2026-02-25T15:07:59.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f78627fc640 10 monclient: _check_auth_tickets 2026-02-25T15:07:59.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f78627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:59.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f787c016640 5 monclient: authenticate success, global_id 30918 2026-02-25T15:07:59.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f787c016640 10 monclient: _renew_subs 2026-02-25T15:07:59.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f787c016640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f787c016640 1 -- 10.20.193.90:0/3969952941 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7874191580 con 0x7f787415e2a0 2026-02-25T15:07:59.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f787c016640 10 client.30918.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:59.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f787c016640 10 monclient: _renew_subs 2026-02-25T15:07:59.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f787c016640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f787c016640 1 -- 10.20.193.90:0/3969952941 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f78740b27d0 con 0x7f787415e2a0 2026-02-25T15:07:59.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f787c016640 1 librados: init done 2026-02-25T15:07:59.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f787c016640 10 client.30918.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:59.131 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f787c016640 15 client.30918.objecter _session_command_op_assign -1 1 2026-02-25T15:07:59.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f787c016640 20 client.30918.objecter _get_session osd=-1 returning homeless 2026-02-25T15:07:59.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f787c016640 10 client.30918.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:59.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f787c016640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.132 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.112+0000 7f787c016640 1 -- 10.20.193.90:0/3969952941 --> v1:10.20.193.90:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f7874195840 con 0x7f787415e2a0 2026-02-25T15:07:59.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.120+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7874195840 con 0x7f787415e2a0 2026-02-25T15:07:59.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.120+0000 7f78627fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:59.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.120+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f7868003700 con 0x7f787415e2a0 2026-02-25T15:07:59.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.120+0000 7f78627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:59.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.120+0000 7f78627fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:59.134 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.120+0000 7f78627fc640 10 monclient: dump: 2026-02-25T15:07:59.134 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.134 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.134 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.134 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.135 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.135 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.135 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.135 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.135 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.135 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.120+0000 7f7863fff640 4 set_mon_vals no callback set 2026-02-25T15:07:59.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.121+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f786801cbe0 con 0x7f787415e2a0 2026-02-25T15:07:59.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.122+0000 7f7879d8b640 10 monclient: get_auth_request con 0x7f784407a110 auth_method 0 2026-02-25T15:07:59.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.122+0000 7f78627fc640 10 client.30918.objecter ms_handle_connect 0x7f784407a110 2026-02-25T15:07:59.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.122+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7f7868094560 con 0x7f787415e2a0 2026-02-25T15:07:59.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.122+0000 7f78627fc640 10 client.30918.objecter ms_dispatch2 0x7f7874000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:59.135 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.122+0000 7f78627fc640 3 client.30918.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:59.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.122+0000 7f78627fc640 3 client.30918.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:59.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.123+0000 7f78627fc640 10 client.30918.objecter checking command 1 2026-02-25T15:07:59.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.123+0000 7f78627fc640 20 client.30918.objecter _get_session s=0x7f78440a43c0 osd=14 3 2026-02-25T15:07:59.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.123+0000 7f78627fc640 20 client.30918.objecter put_session s=0x7f78440a43c0 osd=14 3 2026-02-25T15:07:59.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.123+0000 7f78627fc640 15 client.30918.objecter _session_command_op_remove -1 1 2026-02-25T15:07:59.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.123+0000 7f78627fc640 20 client.30918.objecter _get_session s=0x7f78440a43c0 osd=14 3 2026-02-25T15:07:59.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.123+0000 7f78627fc640 20 client.30918.objecter get_session s=0x7f78440a43c0 osd=14 3 2026-02-25T15:07:59.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.123+0000 7f78627fc640 15 client.30918.objecter _session_command_op_assign 14 1 2026-02-25T15:07:59.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.123+0000 7f78627fc640 20 client.30918.objecter put_session s=0x7f78440a43c0 osd=14 4 2026-02-25T15:07:59.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.123+0000 7f78627fc640 10 client.30918.objecter _send_command 1 2026-02-25T15:07:59.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.123+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 --> v1:10.20.193.94:6808/887271598 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78440c66d0 con 0x7f78440a4a70 2026-02-25T15:07:59.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.123+0000 7f78627fc640 20 client.30918.objecter dump_active .. 0 homeless 2026-02-25T15:07:59.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.123+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_get_version_reply(handle=1 version=338) ==== 24+0+0 (unknown 1001891302 0 0) 0x7f786805d590 con 0x7f787415e2a0 2026-02-25T15:07:59.136 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.123+0000 7f78627fc640 10 monclient: handle_get_version_reply finishing 1 version 338 2026-02-25T15:07:59.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.123+0000 7f787a58c640 10 monclient: get_auth_request con 0x7f78440a4a70 auth_method 0 2026-02-25T15:07:59.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.124+0000 7f78627fc640 10 client.30918.objecter ms_handle_connect 0x7f78440a4a70 2026-02-25T15:07:59.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.133+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== osd.14 v1:10.20.193.94:6808/887271598 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f78440c66d0 con 0x7f78440a4a70 2026-02-25T15:07:59.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.133+0000 7f78627fc640 10 client.30918.objecter ms_dispatch2 0x7f7874000b90 command_reply(tid 1: 0 ) 2026-02-25T15:07:59.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.133+0000 7f78627fc640 10 client.30918.objecter _finish_command 1 = system:0 2026-02-25T15:07:59.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.133+0000 7f78627fc640 20 client.30918.objecter put_session s=0x7f78440a43c0 osd=14 4 2026-02-25T15:07:59.137 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.133+0000 7f78627fc640 15 client.30918.objecter _session_command_op_remove 14 1 2026-02-25T15:07:59.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.140+0000 7f787c016640 10 client.30918.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:07:59.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.140+0000 7f787c016640 15 client.30918.objecter _session_command_op_assign -1 2 2026-02-25T15:07:59.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.140+0000 7f787c016640 20 client.30918.objecter _get_session s=0x7f78440a43c0 osd=14 3 2026-02-25T15:07:59.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.140+0000 7f787c016640 20 client.30918.objecter put_session s=0x7f78440a43c0 osd=14 3 2026-02-25T15:07:59.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.140+0000 7f787c016640 20 client.30918.objecter _get_session s=0x7f78440a43c0 osd=14 3 2026-02-25T15:07:59.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.140+0000 7f787c016640 15 client.30918.objecter _session_command_op_remove -1 2 2026-02-25T15:07:59.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.140+0000 7f787c016640 20 client.30918.objecter get_session s=0x7f78440a43c0 osd=14 3 2026-02-25T15:07:59.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.140+0000 7f787c016640 15 client.30918.objecter _session_command_op_assign 14 2 2026-02-25T15:07:59.142 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.140+0000 7f787c016640 20 client.30918.objecter put_session s=0x7f78440a43c0 osd=14 4 2026-02-25T15:07:59.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.140+0000 7f787c016640 10 client.30918.objecter _send_command 2 2026-02-25T15:07:59.143 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.140+0000 7f787c016640 1 -- 10.20.193.90:0/3969952941 --> v1:10.20.193.94:6808/887271598 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7874192360 con 0x7f78440a4a70 2026-02-25T15:07:59.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.148+0000 7f78627fc640 1 -- 10.20.193.90:0/3969952941 <== osd.14 v1:10.20.193.94:6808/887271598 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f7874192360 con 0x7f78440a4a70 2026-02-25T15:07:59.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.148+0000 7f78627fc640 10 client.30918.objecter ms_dispatch2 0x7f7874000b90 command_reply(tid 2: 0 ) 2026-02-25T15:07:59.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.148+0000 7f78627fc640 10 client.30918.objecter _finish_command 2 = system:0 2026-02-25T15:07:59.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.148+0000 7f78627fc640 20 client.30918.objecter put_session s=0x7f78440a43c0 osd=14 4 2026-02-25T15:07:59.150 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.148+0000 7f78627fc640 15 client.30918.objecter _session_command_op_remove 14 2 2026-02-25T15:07:59.151 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:07:59.151 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-25T15:07:59.151 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:07:59.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 10 librados: watch_flush enter 2026-02-25T15:07:59.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 10 librados: watch_flush exit 2026-02-25T15:07:59.152 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 10 client.30918.objecter close_session for osd.14 2026-02-25T15:07:59.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 1 -- 10.20.193.90:0/3969952941 >> v1:10.20.193.94:6808/887271598 conn(0x7f78440a4a70 legacy=0x7f78440c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 20 client.30918.objecter put_session s=0x7f78440a43c0 osd=14 1 2026-02-25T15:07:59.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 20 client.30918.objecter shutdown clearing up homeless session... 2026-02-25T15:07:59.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 10 client.30918.objecter successfully canceled tick 2026-02-25T15:07:59.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 1 -- 10.20.193.90:0/3969952941 >> v1:10.20.193.189:6800/3647375982 conn(0x7f784407a110 legacy=0x7f784409a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 10 monclient: shutdown 2026-02-25T15:07:59.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:59.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 1 -- 10.20.193.90:0/3969952941 >> v1:10.20.193.90:6789/0 conn(0x7f787415e2a0 legacy=0x7f7874190e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787a58c640 1 -- 10.20.193.90:0/3969952941 reap_dead start 2026-02-25T15:07:59.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 1 -- 10.20.193.90:0/3969952941 shutdown_connections 2026-02-25T15:07:59.153 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 1 -- 10.20.193.90:0/3969952941 >> 10.20.193.90:0/3969952941 conn(0x7f78740b2f60 msgr2=0x7f787415d490 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:59.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 1 -- 10.20.193.90:0/3969952941 shutdown_connections 2026-02-25T15:07:59.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 1 -- 10.20.193.90:0/3969952941 wait complete. 2026-02-25T15:07:59.154 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.149+0000 7f787c016640 1 librados: shutdown 2026-02-25T15:07:59.158 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 14 2026-02-25T15:07:59.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.226+0000 7fa1bea7b640 10 monclient: get_monmap_and_config 2026-02-25T15:07:59.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.226+0000 7fa1bea7b640 10 monclient: build_initial_monmap 2026-02-25T15:07:59.228 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.226+0000 7fa1bea7b640 10 monclient: monmap: 2026-02-25T15:07:59.228 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:59.229 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.229 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:59.227414+0000 2026-02-25T15:07:59.229 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:59.227414+0000 2026-02-25T15:07:59.229 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:59.229 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.229 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:59.229 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:59.229 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:59.229 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.227+0000 7fa1bea7b640 1 Processor -- start 2026-02-25T15:07:59.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.227+0000 7fa1bea7b640 1 -- start start 2026-02-25T15:07:59.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.227+0000 7fa1bea7b640 10 monclient: init 2026-02-25T15:07:59.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.227+0000 7fa1bea7b640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:59.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.227+0000 7fa1bea7b640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:07:59.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.227+0000 7fa1bea7b640 10 monclient(hunting): picked mon.noname-c con 0x7fa1b8157780 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:59.231 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.227+0000 7fa1bea7b640 10 monclient(hunting): picked mon.noname-b con 0x7fa1b8157b50 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:59.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.227+0000 7fa1bea7b640 10 monclient(hunting): picked mon.noname-a con 0x7fa1b8164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:59.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.227+0000 7fa1bea7b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1b805de70 con 0x7fa1b8164d00 2026-02-25T15:07:59.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.227+0000 7fa1bea7b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1b8055b20 con 0x7fa1b8157b50 2026-02-25T15:07:59.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.227+0000 7fa1bea7b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1b8055e00 con 0x7fa1b8157780 2026-02-25T15:07:59.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.227+0000 7fa1bea7b640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:59.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.227+0000 7fa1bea7b640 10 monclient(hunting): authenticate will time out at 3368.827765s 2026-02-25T15:07:59.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.228+0000 7fa1b77fe640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fa1b8157b50 0x7fa1b81645f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45852/0 (socket says 10.20.193.90:45852) 2026-02-25T15:07:59.232 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.228+0000 7fa1b77fe640 1 -- 10.20.193.90:0/912550020 learned_addr learned my addr 10.20.193.90:0/912550020 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:59.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.255+0000 7fa1b6ffd640 1 -- 10.20.193.90:0/912550020 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 546536608 0 0) 0x7fa1b805de70 con 0x7fa1b8164d00 2026-02-25T15:07:59.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.255+0000 7fa1b6ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.255+0000 7fa1b6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.255+0000 7fa1b6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.255+0000 7fa1b6ffd640 10 monclient(hunting): my global_id is 30921 2026-02-25T15:07:59.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.255+0000 7fa1b6ffd640 1 -- 10.20.193.90:0/912550020 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa19c003610 con 0x7fa1b8164d00 2026-02-25T15:07:59.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1b6ffd640 1 -- 10.20.193.90:0/912550020 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3679988879 0 0) 0x7fa19c003610 con 0x7fa1b8164d00 2026-02-25T15:07:59.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1b6ffd640 1 -- 10.20.193.90:0/912550020 >> v1:10.20.193.94:6789/0 conn(0x7fa1b8157780 legacy=0x7fa1b8056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1b6ffd640 1 -- 10.20.193.90:0/912550020 >> v1:10.20.193.93:6789/0 conn(0x7fa1b8157b50 legacy=0x7fa1b81645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1b6ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:07:59.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1b6ffd640 1 monclient: found mon.noname-a 2026-02-25T15:07:59.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1b6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:59.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1b6ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1b6ffd640 1 -- 10.20.193.90:0/912550020 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b80560e0 con 0x7fa1b8164d00 2026-02-25T15:07:59.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1b6ffd640 10 monclient: _finish_auth 0 2026-02-25T15:07:59.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1b6ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:07:59.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:59.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1b6ffd640 1 -- 10.20.193.90:0/912550020 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa1ac003170 con 0x7fa1b8164d00 2026-02-25T15:07:59.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:59.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1b6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:59.260 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1b6ffd640 10 monclient: dump: 2026-02-25T15:07:59.260 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.260 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.260 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.260 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.260 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.260 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.261 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.261 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.261 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.261 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1bea7b640 5 monclient: authenticate success, global_id 30921 2026-02-25T15:07:59.261 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.257+0000 7fa1bea7b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:59.264 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1b6ffd640 1 -- 10.20.193.90:0/912550020 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa1ac003380 con 0x7fa1b8164d00 2026-02-25T15:07:59.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1b6ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:59.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1b6ffd640 1 -- 10.20.193.90:0/912550020 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa1ac003780 con 0x7fa1b8164d00 2026-02-25T15:07:59.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:59.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1b6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:59.265 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1b6ffd640 10 monclient: dump: 2026-02-25T15:07:59.265 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.265 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.265 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.265 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.265 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.266 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.266 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.266 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.266 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.266 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1bea7b640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:59.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1bea7b640 4 set_mon_vals no callback set 2026-02-25T15:07:59.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1bea7b640 10 monclient: shutdown 2026-02-25T15:07:59.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1bea7b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:59.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1bea7b640 1 -- 10.20.193.90:0/912550020 >> v1:10.20.193.90:6789/0 conn(0x7fa1b8164d00 legacy=0x7fa1b81c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.266 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1bea7b640 1 -- 10.20.193.90:0/912550020 shutdown_connections 2026-02-25T15:07:59.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1bea7b640 1 -- 10.20.193.90:0/912550020 >> 10.20.193.90:0/912550020 conn(0x7fa1b80b2ef0 msgr2=0x7fa1b80b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:59.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1bea7b640 1 -- 10.20.193.90:0/912550020 shutdown_connections 2026-02-25T15:07:59.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1bea7b640 1 -- 10.20.193.90:0/912550020 wait complete. 2026-02-25T15:07:59.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1bea7b640 10 monclient: build_initial_monmap 2026-02-25T15:07:59.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1bea7b640 10 monclient: monmap: 2026-02-25T15:07:59.267 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:59.267 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:59.267 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:59.267 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:59.267 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:59.267 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:59.267 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:59.267 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:59.268 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:59.268 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1bea7b640 1 librados: starting msgr at 2026-02-25T15:07:59.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.258+0000 7fa1bea7b640 1 librados: starting objecter 2026-02-25T15:07:59.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:59.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 1 Processor -- start 2026-02-25T15:07:59.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 1 -- start start 2026-02-25T15:07:59.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 1 librados: setting wanted keys 2026-02-25T15:07:59.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 1 librados: calling monclient init 2026-02-25T15:07:59.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 10 monclient: init 2026-02-25T15:07:59.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:59.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:07:59.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 10 monclient(hunting): picked mon.noname-c con 0x7fa1b8157780 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:59.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 10 monclient(hunting): picked mon.noname-a con 0x7fa1b8157b50 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:59.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 10 monclient(hunting): picked mon.noname-b con 0x7fa1b8164d00 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:59.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1b815d890 con 0x7fa1b8157b50 2026-02-25T15:07:59.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1b8129ec0 con 0x7fa1b8164d00 2026-02-25T15:07:59.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1b812a1a0 con 0x7fa1b8157780 2026-02-25T15:07:59.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:59.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1bea7b640 10 monclient(hunting): authenticate will time out at 3368.859469s 2026-02-25T15:07:59.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1b77fe640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fa1b8157b50 0x7fa1b812af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:40040/0 (socket says 10.20.193.90:40040) 2026-02-25T15:07:59.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1b77fe640 1 -- 10.20.193.90:0/1174897294 learned_addr learned my addr 10.20.193.90:0/1174897294 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:59.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1b4ff9640 10 client.?.objecter ms_handle_connect 0x7fa1b8157b50 2026-02-25T15:07:59.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.259+0000 7fa1b4ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:59.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.260+0000 7fa1b4ff9640 1 -- 10.20.193.90:0/1174897294 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 344651394 0 0) 0x7fa1b815d890 con 0x7fa1b8157b50 2026-02-25T15:07:59.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.260+0000 7fa1b4ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.260+0000 7fa1b4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.260+0000 7fa1b4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.260+0000 7fa1b4ff9640 10 monclient(hunting): my global_id is 30924 2026-02-25T15:07:59.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.260+0000 7fa1b4ff9640 1 -- 10.20.193.90:0/1174897294 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa188003610 con 0x7fa1b8157b50 2026-02-25T15:07:59.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.262+0000 7fa1b4ff9640 1 -- 10.20.193.90:0/1174897294 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1360822261 0 0) 0x7fa188003610 con 0x7fa1b8157b50 2026-02-25T15:07:59.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.262+0000 7fa1b4ff9640 1 -- 10.20.193.90:0/1174897294 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa1b815d890 con 0x7fa1b8157b50 2026-02-25T15:07:59.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.262+0000 7fa1b4ff9640 1 -- 10.20.193.90:0/1174897294 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa1a8003010 con 0x7fa1b8157b50 2026-02-25T15:07:59.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.262+0000 7fa1b4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:59.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.262+0000 7fa1b4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:59.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.262+0000 7fa1b4ff9640 10 monclient(hunting): dump: 2026-02-25T15:07:59.270 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.271 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.271 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.271 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.271 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.271 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.271 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.271 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.271 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.271 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.262+0000 7fa1b4ff9640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:59.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1b4ff9640 1 -- 10.20.193.90:0/1174897294 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1703389760 0 0) 0x7fa1b815d890 con 0x7fa1b8157b50 2026-02-25T15:07:59.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1b4ff9640 1 -- 10.20.193.90:0/1174897294 >> v1:10.20.193.94:6789/0 conn(0x7fa1b8157780 legacy=0x7fa1b815d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1b4ff9640 1 -- 10.20.193.90:0/1174897294 >> v1:10.20.193.93:6789/0 conn(0x7fa1b8164d00 legacy=0x7fa1b80aac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1b4ff9640 10 monclient: _finish_hunting 0 2026-02-25T15:07:59.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1b4ff9640 1 monclient: found mon.a 2026-02-25T15:07:59.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1b4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:59.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1b4ff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1b4ff9640 1 -- 10.20.193.90:0/1174897294 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b812a480 con 0x7fa1b8157b50 2026-02-25T15:07:59.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1b4ff9640 10 monclient: _finish_auth 0 2026-02-25T15:07:59.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1b4ff9640 10 monclient: _check_auth_tickets 2026-02-25T15:07:59.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1b4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:59.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1bea7b640 5 monclient: authenticate success, global_id 30924 2026-02-25T15:07:59.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1bea7b640 10 monclient: _renew_subs 2026-02-25T15:07:59.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1bea7b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1bea7b640 1 -- 10.20.193.90:0/1174897294 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b812a760 con 0x7fa1b8157b50 2026-02-25T15:07:59.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1bea7b640 10 client.30924.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:59.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1bea7b640 10 monclient: _renew_subs 2026-02-25T15:07:59.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1bea7b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1bea7b640 1 -- 10.20.193.90:0/1174897294 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa1b812b670 con 0x7fa1b8157b50 2026-02-25T15:07:59.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.264+0000 7fa1bea7b640 1 librados: init done 2026-02-25T15:07:59.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.265+0000 7fa1bea7b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:59.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.265+0000 7fa1bea7b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:59.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.265+0000 7fa1bea7b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.265+0000 7fa1bea7b640 1 -- 10.20.193.90:0/1174897294 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b812b900 con 0x7fa1b8157b50 2026-02-25T15:07:59.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.272+0000 7fa1b4ff9640 1 -- 10.20.193.90:0/1174897294 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa1a80048d0 con 0x7fa1b8157b50 2026-02-25T15:07:59.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.272+0000 7fa1b4ff9640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:59.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.272+0000 7fa1b4ff9640 1 -- 10.20.193.90:0/1174897294 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fa1a8004b40 con 0x7fa1b8157b50 2026-02-25T15:07:59.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.272+0000 7fa1b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:59.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.272+0000 7fa1b4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:59.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.272+0000 7fa1b67fc640 4 set_mon_vals no callback set 2026-02-25T15:07:59.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.272+0000 7fa1b4ff9640 10 monclient: dump: 2026-02-25T15:07:59.277 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.277 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.277 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.277 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.277 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.277 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.277 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.277 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.277 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.278 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.273+0000 7fa1b4ff9640 1 -- 10.20.193.90:0/1174897294 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fa1a8004d20 con 0x7fa1b8157b50 2026-02-25T15:07:59.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.273+0000 7fa1b7fff640 10 monclient: get_auth_request con 0x7fa188079f30 auth_method 0 2026-02-25T15:07:59.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.274+0000 7fa1b4ff9640 10 client.30924.objecter ms_handle_connect 0x7fa188079f30 2026-02-25T15:07:59.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.274+0000 7fa1b4ff9640 1 -- 10.20.193.90:0/1174897294 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7fa1a8095d50 con 0x7fa1b8157b50 2026-02-25T15:07:59.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.274+0000 7fa1b4ff9640 10 client.30924.objecter ms_dispatch2 0x7fa1b8000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:59.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.274+0000 7fa1b4ff9640 3 client.30924.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:59.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.274+0000 7fa1b4ff9640 3 client.30924.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:59.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.274+0000 7fa1b4ff9640 20 client.30924.objecter dump_active .. 0 homeless 2026-02-25T15:07:59.363 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.361+0000 7fa1b4ff9640 1 -- 10.20.193.90:0/1174897294 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fa1a805f6b0 con 0x7fa1b8157b50 2026-02-25T15:07:59.363 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.361+0000 7fa1b4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:59.363 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.361+0000 7fa1b4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:59.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.470+0000 7fa1bea7b640 1 -- 10.20.193.90:0/1174897294 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "14", "target": ["mon-mgr", ""]}) -- 0x7fa1b8129c70 con 0x7fa188079f30 2026-02-25T15:07:59.473 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.471+0000 7fa1b4ff9640 1 -- 10.20.193.90:0/1174897294 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 14 to deep-scrub) ==== 42+0+0 (unknown 1939288360 0 0) 0x7fa1b8129c70 con 0x7fa188079f30 2026-02-25T15:07:59.473 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 14 to deep-scrub 2026-02-25T15:07:59.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.472+0000 7fa1bea7b640 10 librados: watch_flush enter 2026-02-25T15:07:59.474 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.472+0000 7fa1bea7b640 10 librados: watch_flush exit 2026-02-25T15:07:59.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.472+0000 7fa1bea7b640 20 client.30924.objecter shutdown clearing up homeless session... 2026-02-25T15:07:59.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.472+0000 7fa1bea7b640 10 client.30924.objecter successfully canceled tick 2026-02-25T15:07:59.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.472+0000 7fa1bea7b640 1 -- 10.20.193.90:0/1174897294 >> v1:10.20.193.189:6800/3647375982 conn(0x7fa188079f30 legacy=0x7fa18809a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.472+0000 7fa1bea7b640 10 monclient: shutdown 2026-02-25T15:07:59.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.472+0000 7fa1bea7b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:59.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.472+0000 7fa1bea7b640 1 -- 10.20.193.90:0/1174897294 >> v1:10.20.193.90:6789/0 conn(0x7fa1b8157b50 legacy=0x7fa1b812af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.472+0000 7fa1bea7b640 1 -- 10.20.193.90:0/1174897294 shutdown_connections 2026-02-25T15:07:59.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.472+0000 7fa1bea7b640 1 -- 10.20.193.90:0/1174897294 >> 10.20.193.90:0/1174897294 conn(0x7fa1b80b2ef0 msgr2=0x7fa1b80ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:59.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.472+0000 7fa1bea7b640 1 -- 10.20.193.90:0/1174897294 shutdown_connections 2026-02-25T15:07:59.475 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.472+0000 7fa1bea7b640 1 -- 10.20.193.90:0/1174897294 wait complete. 2026-02-25T15:07:59.476 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.472+0000 7fa1bea7b640 1 librados: shutdown 2026-02-25T15:07:59.482 INFO:tasks.ceph:Scrubbing osd.3 2026-02-25T15:07:59.483 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:07:59.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.550+0000 7f480222b640 10 monclient: get_monmap_and_config 2026-02-25T15:07:59.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.550+0000 7f480222b640 10 monclient: build_initial_monmap 2026-02-25T15:07:59.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.550+0000 7f480222b640 10 monclient: monmap: 2026-02-25T15:07:59.552 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:59.552 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.552 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:59.550730+0000 2026-02-25T15:07:59.552 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:59.550730+0000 2026-02-25T15:07:59.552 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:59.552 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.552 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:59.552 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:59.552 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:59.553 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.550+0000 7f480222b640 1 Processor -- start 2026-02-25T15:07:59.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.550+0000 7f480222b640 1 -- start start 2026-02-25T15:07:59.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.550+0000 7f480222b640 10 monclient: init 2026-02-25T15:07:59.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.551+0000 7f480222b640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:59.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.551+0000 7f480222b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:07:59.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.551+0000 7f480222b640 10 monclient(hunting): picked mon.noname-a con 0x7f47fc0b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:59.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.551+0000 7f480222b640 10 monclient(hunting): picked mon.noname-c con 0x7f47fc0ba350 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:59.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.551+0000 7f480222b640 10 monclient(hunting): picked mon.noname-b con 0x7f47fc0585e0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:59.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.551+0000 7f480222b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47fc163930 con 0x7f47fc0b9f80 2026-02-25T15:07:59.555 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.551+0000 7f480222b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47fc163bb0 con 0x7f47fc0585e0 2026-02-25T15:07:59.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.551+0000 7f480222b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47fc163e90 con 0x7f47fc0ba350 2026-02-25T15:07:59.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.551+0000 7f480222b640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:59.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.551+0000 7f480222b640 10 monclient(hunting): authenticate will time out at 3369.151116s 2026-02-25T15:07:59.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.551+0000 7f47fb7fe640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f47fc0b9f80 0x7f47fc163220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:40046/0 (socket says 10.20.193.90:40046) 2026-02-25T15:07:59.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.551+0000 7f47fbfff640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f47fc0585e0 0x7f47fc166cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45864/0 (socket says 10.20.193.90:45864) 2026-02-25T15:07:59.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.551+0000 7f47fb7fe640 1 -- 10.20.193.90:0/4017490485 learned_addr learned my addr 10.20.193.90:0/4017490485 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:59.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.552+0000 7f47fa7fc640 1 -- 10.20.193.90:0/4017490485 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3146806191 0 0) 0x7f47fc163bb0 con 0x7f47fc0585e0 2026-02-25T15:07:59.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.552+0000 7f47fa7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.552+0000 7f47fa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.552+0000 7f47fa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.556 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.552+0000 7f47fa7fc640 10 monclient(hunting): my global_id is 27556 2026-02-25T15:07:59.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.552+0000 7f47fa7fc640 1 -- 10.20.193.90:0/4017490485 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47e0003610 con 0x7f47fc0585e0 2026-02-25T15:07:59.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.552+0000 7f47fa7fc640 1 -- 10.20.193.90:0/4017490485 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 619526645 0 0) 0x7f47fc163e90 con 0x7f47fc0ba350 2026-02-25T15:07:59.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.552+0000 7f47fa7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.552+0000 7f47fa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.552+0000 7f47fa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.552+0000 7f47fa7fc640 10 monclient(hunting): my global_id is 28694 2026-02-25T15:07:59.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.553+0000 7f47fa7fc640 1 -- 10.20.193.90:0/4017490485 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47fc163bb0 con 0x7f47fc0ba350 2026-02-25T15:07:59.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.553+0000 7f47fa7fc640 1 -- 10.20.193.90:0/4017490485 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1038369517 0 0) 0x7f47e8002910 con 0x7f47fc0b9f80 2026-02-25T15:07:59.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.553+0000 7f47fa7fc640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.553+0000 7f47fa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.553+0000 7f47fa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.553+0000 7f47fa7fc640 10 monclient(hunting): my global_id is 30927 2026-02-25T15:07:59.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.553+0000 7f47fa7fc640 1 -- 10.20.193.90:0/4017490485 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47fc163e90 con 0x7f47fc0b9f80 2026-02-25T15:07:59.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f47fa7fc640 1 -- 10.20.193.90:0/4017490485 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4243005308 0 0) 0x7f47e0003610 con 0x7f47fc0585e0 2026-02-25T15:07:59.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f47fa7fc640 1 -- 10.20.193.90:0/4017490485 >> v1:10.20.193.94:6789/0 conn(0x7f47fc0ba350 legacy=0x7f47fc057ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f47fa7fc640 1 -- 10.20.193.90:0/4017490485 >> v1:10.20.193.90:6789/0 conn(0x7f47fc0b9f80 legacy=0x7f47fc163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f47fa7fc640 10 monclient: _finish_hunting 0 2026-02-25T15:07:59.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f47fa7fc640 1 monclient: found mon.noname-b 2026-02-25T15:07:59.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f47fa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:59.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f47fa7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:07:59.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f47fa7fc640 1 -- 10.20.193.90:0/4017490485 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47fc0573f0 con 0x7f47fc0585e0 2026-02-25T15:07:59.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f47fa7fc640 10 monclient: _finish_auth 0 2026-02-25T15:07:59.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f47fa7fc640 10 monclient: _check_auth_tickets 2026-02-25T15:07:59.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f47fa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:59.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f47fa7fc640 1 -- 10.20.193.90:0/4017490485 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f47f0003170 con 0x7f47fc0585e0 2026-02-25T15:07:59.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f47fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:59.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f47fa7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:59.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f47fa7fc640 10 monclient: dump: 2026-02-25T15:07:59.564 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.564 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.564 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.564 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.564 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.564 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.564 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.564 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.564 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.564 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f480222b640 5 monclient: authenticate success, global_id 27556 2026-02-25T15:07:59.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.554+0000 7f480222b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:59.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.555+0000 7f47fa7fc640 1 -- 10.20.193.90:0/4017490485 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f47f0003340 con 0x7f47fc0585e0 2026-02-25T15:07:59.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.555+0000 7f47fa7fc640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:59.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.555+0000 7f47fa7fc640 1 -- 10.20.193.90:0/4017490485 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f47f0003780 con 0x7f47fc0585e0 2026-02-25T15:07:59.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.555+0000 7f47fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:59.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.555+0000 7f47fa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:59.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.555+0000 7f47fa7fc640 10 monclient: dump: 2026-02-25T15:07:59.565 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.565 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.565 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.565 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.565 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.565 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.566 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.566 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.566 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.566 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.555+0000 7f480222b640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:59.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.555+0000 7f480222b640 4 set_mon_vals no callback set 2026-02-25T15:07:59.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.555+0000 7f480222b640 10 monclient: shutdown 2026-02-25T15:07:59.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.555+0000 7f480222b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:59.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.555+0000 7f480222b640 1 -- 10.20.193.90:0/4017490485 >> v1:10.20.193.93:6789/0 conn(0x7f47fc0585e0 legacy=0x7f47fc166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.555+0000 7f480222b640 1 -- 10.20.193.90:0/4017490485 shutdown_connections 2026-02-25T15:07:59.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.555+0000 7f480222b640 1 -- 10.20.193.90:0/4017490485 >> 10.20.193.90:0/4017490485 conn(0x7f47fc0b7990 msgr2=0x7f47fc0b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:59.566 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.555+0000 7f480222b640 1 -- 10.20.193.90:0/4017490485 shutdown_connections 2026-02-25T15:07:59.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.555+0000 7f480222b640 1 -- 10.20.193.90:0/4017490485 wait complete. 2026-02-25T15:07:59.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.556+0000 7f480222b640 10 monclient: build_initial_monmap 2026-02-25T15:07:59.567 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.556+0000 7f480222b640 10 monclient: monmap: 2026-02-25T15:07:59.567 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:59.567 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:59.567 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:59.567 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:59.567 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:59.567 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:59.567 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:59.567 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:59.567 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:59.568 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.556+0000 7f480222b640 1 librados: starting msgr at 2026-02-25T15:07:59.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.556+0000 7f480222b640 1 librados: starting objecter 2026-02-25T15:07:59.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.556+0000 7f480222b640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:59.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.556+0000 7f480222b640 1 Processor -- start 2026-02-25T15:07:59.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f480222b640 1 -- start start 2026-02-25T15:07:59.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f480222b640 1 librados: setting wanted keys 2026-02-25T15:07:59.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f480222b640 1 librados: calling monclient init 2026-02-25T15:07:59.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f480222b640 10 monclient: init 2026-02-25T15:07:59.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f480222b640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:59.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f480222b640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:07:59.568 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f480222b640 10 monclient(hunting): picked mon.noname-c con 0x7f47fc0585e0 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:59.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f480222b640 10 monclient(hunting): picked mon.noname-a con 0x7f47fc0b9f80 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:59.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f480222b640 10 monclient(hunting): picked mon.noname-b con 0x7f47fc0ba350 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:59.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f480222b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47fc166580 con 0x7f47fc0b9f80 2026-02-25T15:07:59.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f480222b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47fc162f20 con 0x7f47fc0ba350 2026-02-25T15:07:59.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f480222b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47fc166840 con 0x7f47fc0585e0 2026-02-25T15:07:59.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f480222b640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:59.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f480222b640 10 monclient(hunting): authenticate will time out at 3369.157576s 2026-02-25T15:07:59.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f47faffd640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f47fc0b9f80 0x7f47fc162810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:40048/0 (socket says 10.20.193.90:40048) 2026-02-25T15:07:59.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.557+0000 7f47faffd640 1 -- 10.20.193.90:0/524458791 learned_addr learned my addr 10.20.193.90:0/524458791 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:59.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.558+0000 7f47d7fff640 10 client.?.objecter ms_handle_connect 0x7f47fc0b9f80 2026-02-25T15:07:59.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.558+0000 7f47d7fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:59.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.558+0000 7f47d7fff640 10 client.?.objecter ms_handle_connect 0x7f47fc0ba350 2026-02-25T15:07:59.569 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.558+0000 7f47d7fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:59.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.558+0000 7f47d7fff640 10 client.?.objecter ms_handle_connect 0x7f47fc0585e0 2026-02-25T15:07:59.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.558+0000 7f47d7fff640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:59.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 980823791 0 0) 0x7f47fc166580 con 0x7f47fc0b9f80 2026-02-25T15:07:59.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 10 monclient(hunting): my global_id is 30930 2026-02-25T15:07:59.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47cc003610 con 0x7f47fc0b9f80 2026-02-25T15:07:59.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 780227380 0 0) 0x7f47fc166840 con 0x7f47fc0585e0 2026-02-25T15:07:59.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 10 monclient(hunting): my global_id is 28697 2026-02-25T15:07:59.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47fc166580 con 0x7f47fc0585e0 2026-02-25T15:07:59.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2700611065 0 0) 0x7f47fc162f20 con 0x7f47fc0ba350 2026-02-25T15:07:59.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 10 monclient(hunting): my global_id is 27559 2026-02-25T15:07:59.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.559+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47fc166840 con 0x7f47fc0ba350 2026-02-25T15:07:59.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.561+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2862872750 0 0) 0x7f47cc003610 con 0x7f47fc0b9f80 2026-02-25T15:07:59.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.561+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f47fc162f20 con 0x7f47fc0b9f80 2026-02-25T15:07:59.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.561+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f47ec0030f0 con 0x7f47fc0b9f80 2026-02-25T15:07:59.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.561+0000 7f47d7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:59.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.561+0000 7f47d7fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:59.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.561+0000 7f47d7fff640 10 monclient(hunting): dump: 2026-02-25T15:07:59.575 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.575 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.575 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.575 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.575 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.575 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.575 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.575 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.576 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.576 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.561+0000 7f47d7fff640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:59.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.562+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1576087391 0 0) 0x7f47fc166840 con 0x7f47fc0ba350 2026-02-25T15:07:59.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.562+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f47cc003610 con 0x7f47fc0ba350 2026-02-25T15:07:59.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.562+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f47f00034f0 con 0x7f47fc0ba350 2026-02-25T15:07:59.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.562+0000 7f47d7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:59.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.562+0000 7f47d7fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:59.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.562+0000 7f47d7fff640 10 monclient(hunting): dump: 2026-02-25T15:07:59.576 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.576 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.576 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.577 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.577 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.577 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.577 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.577 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.577 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.577 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.562+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3808242213 0 0) 0x7f47fc166580 con 0x7f47fc0585e0 2026-02-25T15:07:59.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.562+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f47fc166840 con 0x7f47fc0585e0 2026-02-25T15:07:59.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.562+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f47e80031e0 con 0x7f47fc0585e0 2026-02-25T15:07:59.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.562+0000 7f47d7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:59.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.562+0000 7f47d7fff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:59.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.562+0000 7f47d7fff640 10 monclient(hunting): dump: 2026-02-25T15:07:59.578 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.578 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.578 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.578 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.578 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.578 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.578 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.578 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.578 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.578 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3292174556 0 0) 0x7f47fc162f20 con 0x7f47fc0b9f80 2026-02-25T15:07:59.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 >> v1:10.20.193.94:6789/0 conn(0x7f47fc0585e0 legacy=0x7f47fc165e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 >> v1:10.20.193.93:6789/0 conn(0x7f47fc0ba350 legacy=0x7f47fc1fba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f47d7fff640 10 monclient: _finish_hunting 0 2026-02-25T15:07:59.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f47d7fff640 1 monclient: found mon.a 2026-02-25T15:07:59.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f47d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:59.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f47d7fff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47fc166b20 con 0x7f47fc0b9f80 2026-02-25T15:07:59.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f47d7fff640 10 monclient: _finish_auth 0 2026-02-25T15:07:59.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f47d7fff640 10 monclient: _check_auth_tickets 2026-02-25T15:07:59.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f47d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:59.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f480222b640 5 monclient: authenticate success, global_id 30930 2026-02-25T15:07:59.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f480222b640 10 monclient: _renew_subs 2026-02-25T15:07:59.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f480222b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f480222b640 1 -- 10.20.193.90:0/524458791 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f47fc166e00 con 0x7f47fc0b9f80 2026-02-25T15:07:59.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f480222b640 10 client.30930.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:59.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f480222b640 10 monclient: _renew_subs 2026-02-25T15:07:59.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f480222b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f480222b640 1 -- 10.20.193.90:0/524458791 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f47fc1fc7c0 con 0x7f47fc0b9f80 2026-02-25T15:07:59.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f480222b640 1 librados: init done 2026-02-25T15:07:59.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f480222b640 10 client.30930.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:59.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f480222b640 15 client.30930.objecter _session_command_op_assign -1 1 2026-02-25T15:07:59.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f480222b640 20 client.30930.objecter _get_session osd=-1 returning homeless 2026-02-25T15:07:59.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f480222b640 10 client.30930.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:59.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f480222b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.563+0000 7f480222b640 1 -- 10.20.193.90:0/524458791 --> v1:10.20.193.90:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f47fc1572e0 con 0x7f47fc0b9f80 2026-02-25T15:07:59.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.571+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f47fc1572e0 con 0x7f47fc0b9f80 2026-02-25T15:07:59.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.571+0000 7f47d7fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:59.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.571+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f47ec0039b0 con 0x7f47fc0b9f80 2026-02-25T15:07:59.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.571+0000 7f47d7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:59.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.571+0000 7f47d7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:59.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.571+0000 7f47f9ffb640 4 set_mon_vals no callback set 2026-02-25T15:07:59.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.571+0000 7f47d7fff640 10 monclient: dump: 2026-02-25T15:07:59.581 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.581 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.581 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.581 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.582 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.582 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.582 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.582 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.582 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.582 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.582 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.572+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f47ec003bc0 con 0x7f47fc0b9f80 2026-02-25T15:07:59.582 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.572+0000 7f47fb7fe640 10 monclient: get_auth_request con 0x7f47cc079ff0 auth_method 0 2026-02-25T15:07:59.582 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 10 client.30930.objecter ms_handle_connect 0x7f47cc079ff0 2026-02-25T15:07:59.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7f47ec0039b0 con 0x7f47fc0b9f80 2026-02-25T15:07:59.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 10 client.30930.objecter ms_dispatch2 0x7f47fc000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:59.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 3 client.30930.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:59.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 3 client.30930.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:59.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 10 client.30930.objecter checking command 1 2026-02-25T15:07:59.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 20 client.30930.objecter _get_session s=0x7f47cc0a42a0 osd=3 3 2026-02-25T15:07:59.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 20 client.30930.objecter put_session s=0x7f47cc0a42a0 osd=3 3 2026-02-25T15:07:59.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 15 client.30930.objecter _session_command_op_remove -1 1 2026-02-25T15:07:59.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 20 client.30930.objecter _get_session s=0x7f47cc0a42a0 osd=3 3 2026-02-25T15:07:59.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 20 client.30930.objecter get_session s=0x7f47cc0a42a0 osd=3 3 2026-02-25T15:07:59.584 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 15 client.30930.objecter _session_command_op_assign 3 1 2026-02-25T15:07:59.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 20 client.30930.objecter put_session s=0x7f47cc0a42a0 osd=3 4 2026-02-25T15:07:59.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 10 client.30930.objecter _send_command 1 2026-02-25T15:07:59.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 --> v1:10.20.193.189:6813/2702305154 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47cc0c65b0 con 0x7f47cc0a4950 2026-02-25T15:07:59.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 20 client.30930.objecter dump_active .. 0 homeless 2026-02-25T15:07:59.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_get_version_reply(handle=1 version=338) ==== 24+0+0 (unknown 1001891302 0 0) 0x7f47ec05af10 con 0x7f47fc0b9f80 2026-02-25T15:07:59.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.573+0000 7f47d7fff640 10 monclient: handle_get_version_reply finishing 1 version 338 2026-02-25T15:07:59.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.574+0000 7f47fbfff640 10 monclient: get_auth_request con 0x7f47cc0a4950 auth_method 0 2026-02-25T15:07:59.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.575+0000 7f47d7fff640 10 client.30930.objecter ms_handle_connect 0x7f47cc0a4950 2026-02-25T15:07:59.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.585+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== osd.3 v1:10.20.193.189:6813/2702305154 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7f47cc0c65b0 con 0x7f47cc0a4950 2026-02-25T15:07:59.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.585+0000 7f47d7fff640 10 client.30930.objecter ms_dispatch2 0x7f47fc000b90 command_reply(tid 1: 0 ) 2026-02-25T15:07:59.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.585+0000 7f47d7fff640 10 client.30930.objecter _finish_command 1 = system:0 2026-02-25T15:07:59.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.585+0000 7f47d7fff640 20 client.30930.objecter put_session s=0x7f47cc0a42a0 osd=3 4 2026-02-25T15:07:59.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.585+0000 7f47d7fff640 15 client.30930.objecter _session_command_op_remove 3 1 2026-02-25T15:07:59.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.593+0000 7f480222b640 10 client.30930.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:07:59.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.593+0000 7f480222b640 15 client.30930.objecter _session_command_op_assign -1 2 2026-02-25T15:07:59.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.593+0000 7f480222b640 20 client.30930.objecter _get_session s=0x7f47cc0a42a0 osd=3 3 2026-02-25T15:07:59.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.593+0000 7f480222b640 20 client.30930.objecter put_session s=0x7f47cc0a42a0 osd=3 3 2026-02-25T15:07:59.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.593+0000 7f480222b640 20 client.30930.objecter _get_session s=0x7f47cc0a42a0 osd=3 3 2026-02-25T15:07:59.595 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.593+0000 7f480222b640 15 client.30930.objecter _session_command_op_remove -1 2 2026-02-25T15:07:59.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.593+0000 7f480222b640 20 client.30930.objecter get_session s=0x7f47cc0a42a0 osd=3 3 2026-02-25T15:07:59.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.593+0000 7f480222b640 15 client.30930.objecter _session_command_op_assign 3 2 2026-02-25T15:07:59.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.593+0000 7f480222b640 20 client.30930.objecter put_session s=0x7f47cc0a42a0 osd=3 4 2026-02-25T15:07:59.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.593+0000 7f480222b640 10 client.30930.objecter _send_command 2 2026-02-25T15:07:59.596 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.593+0000 7f480222b640 1 -- 10.20.193.90:0/524458791 --> v1:10.20.193.189:6813/2702305154 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f47fc159e60 con 0x7f47cc0a4950 2026-02-25T15:07:59.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f47d7fff640 1 -- 10.20.193.90:0/524458791 <== osd.3 v1:10.20.193.189:6813/2702305154 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f47fc159e60 con 0x7f47cc0a4950 2026-02-25T15:07:59.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f47d7fff640 10 client.30930.objecter ms_dispatch2 0x7f47fc000b90 command_reply(tid 2: 0 ) 2026-02-25T15:07:59.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f47d7fff640 10 client.30930.objecter _finish_command 2 = system:0 2026-02-25T15:07:59.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f47d7fff640 20 client.30930.objecter put_session s=0x7f47cc0a42a0 osd=3 4 2026-02-25T15:07:59.603 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f47d7fff640 15 client.30930.objecter _session_command_op_remove 3 2 2026-02-25T15:07:59.603 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:07:59.604 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-25T15:07:59.604 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:07:59.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 10 librados: watch_flush enter 2026-02-25T15:07:59.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 10 librados: watch_flush exit 2026-02-25T15:07:59.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 10 client.30930.objecter close_session for osd.3 2026-02-25T15:07:59.605 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 1 -- 10.20.193.90:0/524458791 >> v1:10.20.193.189:6813/2702305154 conn(0x7f47cc0a4950 legacy=0x7f47cc0c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 20 client.30930.objecter put_session s=0x7f47cc0a42a0 osd=3 1 2026-02-25T15:07:59.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 20 client.30930.objecter shutdown clearing up homeless session... 2026-02-25T15:07:59.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 10 client.30930.objecter successfully canceled tick 2026-02-25T15:07:59.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 1 -- 10.20.193.90:0/524458791 >> v1:10.20.193.189:6800/3647375982 conn(0x7f47cc079ff0 legacy=0x7f47cc09a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 10 monclient: shutdown 2026-02-25T15:07:59.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:59.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 1 -- 10.20.193.90:0/524458791 >> v1:10.20.193.90:6789/0 conn(0x7f47fc0b9f80 legacy=0x7f47fc162810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f47fbfff640 1 -- 10.20.193.90:0/524458791 reap_dead start 2026-02-25T15:07:59.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 1 -- 10.20.193.90:0/524458791 shutdown_connections 2026-02-25T15:07:59.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 1 -- 10.20.193.90:0/524458791 >> 10.20.193.90:0/524458791 conn(0x7f47fc0b7990 msgr2=0x7f47fc15b460 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:59.606 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 1 -- 10.20.193.90:0/524458791 shutdown_connections 2026-02-25T15:07:59.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 1 -- 10.20.193.90:0/524458791 wait complete. 2026-02-25T15:07:59.607 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.601+0000 7f480222b640 1 librados: shutdown 2026-02-25T15:07:59.611 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2026-02-25T15:07:59.677 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.675+0000 7f6fd72b1640 10 monclient: get_monmap_and_config 2026-02-25T15:07:59.677 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.675+0000 7f6fd72b1640 10 monclient: build_initial_monmap 2026-02-25T15:07:59.677 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.675+0000 7f6fd72b1640 10 monclient: monmap: 2026-02-25T15:07:59.677 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:59.677 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.677 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:59.676438+0000 2026-02-25T15:07:59.677 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:59.676438+0000 2026-02-25T15:07:59.677 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:59.677 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.677 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:59.677 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:59.677 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:59.677 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.676+0000 7f6fd72b1640 1 Processor -- start 2026-02-25T15:07:59.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.676+0000 7f6fd72b1640 1 -- start start 2026-02-25T15:07:59.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.676+0000 7f6fd72b1640 10 monclient: init 2026-02-25T15:07:59.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.676+0000 7f6fd72b1640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:59.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.676+0000 7f6fd72b1640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:07:59.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.676+0000 7f6fd72b1640 10 monclient(hunting): picked mon.noname-c con 0x7f6fd0157780 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:59.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.676+0000 7f6fd72b1640 10 monclient(hunting): picked mon.noname-a con 0x7f6fd0157b50 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:59.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.676+0000 7f6fd72b1640 10 monclient(hunting): picked mon.noname-b con 0x7f6fd0164d00 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:59.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.676+0000 7f6fd72b1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd005de70 con 0x7f6fd0157b50 2026-02-25T15:07:59.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.676+0000 7f6fd72b1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd0055b20 con 0x7f6fd0164d00 2026-02-25T15:07:59.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.676+0000 7f6fd72b1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd0055e00 con 0x7f6fd0157780 2026-02-25T15:07:59.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.676+0000 7f6fd72b1640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:59.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.676+0000 7f6fd72b1640 10 monclient(hunting): authenticate will time out at 3369.276795s 2026-02-25T15:07:59.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.677+0000 7f6fd4825640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f6fd0157b50 0x7f6fd01645f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:40058/0 (socket says 10.20.193.90:40058) 2026-02-25T15:07:59.678 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.677+0000 7f6fd4825640 1 -- 10.20.193.90:0/4124758894 learned_addr learned my addr 10.20.193.90:0/4124758894 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:59.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 1 -- 10.20.193.90:0/4124758894 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 752820189 0 0) 0x7f6fd005de70 con 0x7f6fd0157b50 2026-02-25T15:07:59.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 10 monclient(hunting): my global_id is 30933 2026-02-25T15:07:59.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 1 -- 10.20.193.90:0/4124758894 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fb8003610 con 0x7f6fd0157b50 2026-02-25T15:07:59.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 1 -- 10.20.193.90:0/4124758894 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3311480291 0 0) 0x7f6fd0055b20 con 0x7f6fd0164d00 2026-02-25T15:07:59.679 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 10 monclient(hunting): my global_id is 27562 2026-02-25T15:07:59.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 1 -- 10.20.193.90:0/4124758894 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fd005de70 con 0x7f6fd0164d00 2026-02-25T15:07:59.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 1 -- 10.20.193.90:0/4124758894 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1202137427 0 0) 0x7f6fd0055e00 con 0x7f6fd0157780 2026-02-25T15:07:59.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 10 monclient(hunting): my global_id is 28700 2026-02-25T15:07:59.680 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.678+0000 7f6fbffff640 1 -- 10.20.193.90:0/4124758894 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fd0055b20 con 0x7f6fd0157780 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fbffff640 1 -- 10.20.193.90:0/4124758894 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3785649844 0 0) 0x7f6fd005de70 con 0x7f6fd0164d00 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fbffff640 1 -- 10.20.193.90:0/4124758894 >> v1:10.20.193.94:6789/0 conn(0x7f6fd0157780 legacy=0x7f6fd0056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fbffff640 1 -- 10.20.193.90:0/4124758894 >> v1:10.20.193.90:6789/0 conn(0x7f6fd0157b50 legacy=0x7f6fd01645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fbffff640 10 monclient: _finish_hunting 0 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fbffff640 1 monclient: found mon.noname-b 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fbffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fbffff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fbffff640 1 -- 10.20.193.90:0/4124758894 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd00560e0 con 0x7f6fd0164d00 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fbffff640 10 monclient: _finish_auth 0 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fbffff640 10 monclient: _check_auth_tickets 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fbffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fbffff640 1 -- 10.20.193.90:0/4124758894 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6fcc003170 con 0x7f6fd0164d00 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fbffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fbffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fbffff640 10 monclient: dump: 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.681 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fd72b1640 5 monclient: authenticate success, global_id 27562 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.680+0000 7f6fd72b1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fbffff640 1 -- 10.20.193.90:0/4124758894 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6fcc0033c0 con 0x7f6fd0164d00 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fbffff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fbffff640 1 -- 10.20.193.90:0/4124758894 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6fcc003800 con 0x7f6fd0164d00 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fbffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fbffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fbffff640 10 monclient: dump: 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.682 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fd72b1640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fd72b1640 4 set_mon_vals no callback set 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fd72b1640 10 monclient: shutdown 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fd72b1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fd72b1640 1 -- 10.20.193.90:0/4124758894 >> v1:10.20.193.93:6789/0 conn(0x7f6fd0164d00 legacy=0x7f6fd01c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fd72b1640 1 -- 10.20.193.90:0/4124758894 shutdown_connections 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fd72b1640 1 -- 10.20.193.90:0/4124758894 >> 10.20.193.90:0/4124758894 conn(0x7f6fd00b2ef0 msgr2=0x7f6fd00b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fd72b1640 1 -- 10.20.193.90:0/4124758894 shutdown_connections 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fd72b1640 1 -- 10.20.193.90:0/4124758894 wait complete. 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.681+0000 7f6fd72b1640 10 monclient: build_initial_monmap 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 10 monclient: monmap: 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:59.683 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 1 librados: starting msgr at 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 1 librados: starting objecter 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 1 Processor -- start 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 1 -- start start 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 1 librados: setting wanted keys 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 1 librados: calling monclient init 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 10 monclient: init 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 10 monclient(hunting): picked mon.noname-c con 0x7f6fd0157780 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 10 monclient(hunting): picked mon.noname-b con 0x7f6fd0157b50 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 10 monclient(hunting): picked mon.noname-a con 0x7f6fd0164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd00ae5d0 con 0x7f6fd0164d00 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd00ae8b0 con 0x7f6fd0157b50 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd00aeb90 con 0x7f6fd0157780 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.682+0000 7f6fd72b1640 10 monclient(hunting): authenticate will time out at 3369.282765s 2026-02-25T15:07:59.684 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.683+0000 7f6fd5827640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f6fd0164d00 0x7f6fd0123100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:40066/0 (socket says 10.20.193.90:40066) 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.683+0000 7f6fd5827640 1 -- 10.20.193.90:0/1651761381 learned_addr learned my addr 10.20.193.90:0/1651761381 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.683+0000 7f6fbdffb640 10 client.?.objecter ms_handle_connect 0x7f6fd0164d00 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.683+0000 7f6fbdffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.683+0000 7f6fbdffb640 10 client.?.objecter ms_handle_connect 0x7f6fd0157780 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.683+0000 7f6fbdffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.683+0000 7f6fbdffb640 10 client.?.objecter ms_handle_connect 0x7f6fd0157b50 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.683+0000 7f6fbdffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3061140902 0 0) 0x7f6fd00aeb90 con 0x7f6fd0157780 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 10 monclient(hunting): my global_id is 28703 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fa0003610 con 0x7f6fd0157780 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 843761974 0 0) 0x7f6fd00ae5d0 con 0x7f6fd0164d00 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 10 monclient(hunting): my global_id is 30936 2026-02-25T15:07:59.685 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fd00aeb90 con 0x7f6fd0164d00 2026-02-25T15:07:59.686 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 471290985 0 0) 0x7f6fd00ae8b0 con 0x7f6fd0157b50 2026-02-25T15:07:59.686 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.686 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.686 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.686 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 10 monclient(hunting): my global_id is 27565 2026-02-25T15:07:59.686 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.684+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fd00ae5d0 con 0x7f6fd0157b50 2026-02-25T15:07:59.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.686+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2411316271 0 0) 0x7f6fd00aeb90 con 0x7f6fd0164d00 2026-02-25T15:07:59.687 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.686+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6fd00ae8b0 con 0x7f6fd0164d00 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.686+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6fcc003710 con 0x7f6fd0164d00 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.686+0000 7f6fbdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.686+0000 7f6fbdffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.686+0000 7f6fbdffb640 10 monclient(hunting): dump: 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.686+0000 7f6fbdffb640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.686+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4127698778 0 0) 0x7f6fa0003610 con 0x7f6fd0157780 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.686+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6fd00aeb90 con 0x7f6fd0157780 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.686+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6fc0003060 con 0x7f6fd0157780 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.686+0000 7f6fbdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.686+0000 7f6fbdffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.686+0000 7f6fbdffb640 10 monclient(hunting): dump: 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.688 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.687+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1617848101 0 0) 0x7f6fd00ae5d0 con 0x7f6fd0157b50 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.687+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6fa0003610 con 0x7f6fd0157b50 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.687+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6fc40030b0 con 0x7f6fd0157b50 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.687+0000 7f6fbdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.687+0000 7f6fbdffb640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.687+0000 7f6fbdffb640 10 monclient(hunting): dump: 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.689 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1134227394 0 0) 0x7f6fd00ae8b0 con 0x7f6fd0164d00 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 >> v1:10.20.193.94:6789/0 conn(0x7f6fd0157780 legacy=0x7f6fd015bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 >> v1:10.20.193.93:6789/0 conn(0x7f6fd0157b50 legacy=0x7f6fd013a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fbdffb640 10 monclient: _finish_hunting 0 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fbdffb640 1 monclient: found mon.a 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fbdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fbdffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd00aee70 con 0x7f6fd0164d00 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fbdffb640 10 monclient: _finish_auth 0 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fbdffb640 10 monclient: _check_auth_tickets 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fbdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fd72b1640 5 monclient: authenticate success, global_id 30936 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fd72b1640 10 monclient: _renew_subs 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fd72b1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fd72b1640 1 -- 10.20.193.90:0/1651761381 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd00af150 con 0x7f6fd0164d00 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fd72b1640 10 client.30936.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fd72b1640 10 monclient: _renew_subs 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fd72b1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fd72b1640 1 -- 10.20.193.90:0/1651761381 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f6fd0156ff0 con 0x7f6fd0164d00 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fd72b1640 1 librados: init done 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fd72b1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fd72b1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fd72b1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:07:59.690 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.688+0000 7f6fd72b1640 1 -- 10.20.193.90:0/1651761381 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd015b8f0 con 0x7f6fd0164d00 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.783+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6fcc0033a0 con 0x7f6fd0164d00 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.783+0000 7f6fbdffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.783+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6fcc003c90 con 0x7f6fd0164d00 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.783+0000 7f6fbdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.783+0000 7f6fbdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.783+0000 7f6fbf7fe640 4 set_mon_vals no callback set 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.783+0000 7f6fbdffb640 10 monclient: dump: 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.785 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.784+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f6fcc003c90 con 0x7f6fd0164d00 2026-02-25T15:07:59.786 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.785+0000 7f6fd5026640 10 monclient: get_auth_request con 0x7f6fa007a0d0 auth_method 0 2026-02-25T15:07:59.786 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.785+0000 7f6fbdffb640 10 client.30936.objecter ms_handle_connect 0x7f6fa007a0d0 2026-02-25T15:07:59.788 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.787+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7f6fcc095d10 con 0x7f6fd0164d00 2026-02-25T15:07:59.788 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.787+0000 7f6fbdffb640 10 client.30936.objecter ms_dispatch2 0x7f6fd0000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:07:59.788 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.787+0000 7f6fbdffb640 3 client.30936.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:07:59.788 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.787+0000 7f6fbdffb640 3 client.30936.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:07:59.788 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.787+0000 7f6fbdffb640 20 client.30936.objecter dump_active .. 0 homeless 2026-02-25T15:07:59.788 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.787+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f6fcc05f670 con 0x7f6fd0164d00 2026-02-25T15:07:59.788 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.787+0000 7f6fbdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:07:59.788 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.787+0000 7f6fbdffb640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:07:59.893 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.891+0000 7f6fd72b1640 1 -- 10.20.193.90:0/1651761381 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f6fd00af390 con 0x7f6fa007a0d0 2026-02-25T15:07:59.893 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.891+0000 7f6fbdffb640 1 -- 10.20.193.90:0/1651761381 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (unknown 3526570807 0 0) 0x7f6fd00af390 con 0x7f6fa007a0d0 2026-02-25T15:07:59.893 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 3 to deep-scrub 2026-02-25T15:07:59.895 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.892+0000 7f6fd72b1640 10 librados: watch_flush enter 2026-02-25T15:07:59.895 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.892+0000 7f6fd72b1640 10 librados: watch_flush exit 2026-02-25T15:07:59.895 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.892+0000 7f6fd72b1640 20 client.30936.objecter shutdown clearing up homeless session... 2026-02-25T15:07:59.895 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.892+0000 7f6fd72b1640 10 client.30936.objecter successfully canceled tick 2026-02-25T15:07:59.895 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.892+0000 7f6fd72b1640 1 -- 10.20.193.90:0/1651761381 >> v1:10.20.193.189:6800/3647375982 conn(0x7f6fa007a0d0 legacy=0x7f6fa009a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.895 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.892+0000 7f6fd72b1640 10 monclient: shutdown 2026-02-25T15:07:59.895 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.892+0000 7f6fd72b1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:59.895 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.892+0000 7f6fd72b1640 1 -- 10.20.193.90:0/1651761381 >> v1:10.20.193.90:6789/0 conn(0x7f6fd0164d00 legacy=0x7f6fd0123100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.896 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.892+0000 7f6fd72b1640 1 -- 10.20.193.90:0/1651761381 shutdown_connections 2026-02-25T15:07:59.896 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.892+0000 7f6fd72b1640 1 -- 10.20.193.90:0/1651761381 >> 10.20.193.90:0/1651761381 conn(0x7f6fd00b2ef0 msgr2=0x7f6fd00b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:59.896 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.892+0000 7f6fd72b1640 1 -- 10.20.193.90:0/1651761381 shutdown_connections 2026-02-25T15:07:59.896 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.892+0000 7f6fd72b1640 1 -- 10.20.193.90:0/1651761381 wait complete. 2026-02-25T15:07:59.896 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.892+0000 7f6fd72b1640 1 librados: shutdown 2026-02-25T15:07:59.902 INFO:tasks.ceph:Scrubbing osd.7 2026-02-25T15:07:59.903 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:07:59.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.972+0000 7fb82cdc8640 10 monclient: get_monmap_and_config 2026-02-25T15:07:59.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.972+0000 7fb82cdc8640 10 monclient: build_initial_monmap 2026-02-25T15:07:59.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.972+0000 7fb82cdc8640 10 monclient: monmap: 2026-02-25T15:07:59.975 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:59.975 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.975 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:07:59.973090+0000 2026-02-25T15:07:59.975 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:07:59.973090+0000 2026-02-25T15:07:59.975 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:59.975 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.975 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:59.975 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:59.975 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:59.975 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.975 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.972+0000 7fb82cdc8640 1 Processor -- start 2026-02-25T15:07:59.975 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.972+0000 7fb82cdc8640 1 -- start start 2026-02-25T15:07:59.976 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.972+0000 7fb82cdc8640 10 monclient: init 2026-02-25T15:07:59.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.973+0000 7fb82cdc8640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:59.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.973+0000 7fb82cdc8640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:07:59.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.973+0000 7fb82cdc8640 10 monclient(hunting): picked mon.noname-a con 0x7fb82815e2a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:59.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.973+0000 7fb82cdc8640 10 monclient(hunting): picked mon.noname-c con 0x7fb828059e10 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:59.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.973+0000 7fb82cdc8640 10 monclient(hunting): picked mon.noname-b con 0x7fb8281a5660 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:59.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.973+0000 7fb82cdc8640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb828162df0 con 0x7fb82815e2a0 2026-02-25T15:07:59.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.973+0000 7fb82cdc8640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8281630d0 con 0x7fb8281a5660 2026-02-25T15:07:59.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.973+0000 7fb82cdc8640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8281633b0 con 0x7fb828059e10 2026-02-25T15:07:59.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.973+0000 7fb82cdc8640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:59.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.973+0000 7fb82cdc8640 10 monclient(hunting): authenticate will time out at 3369.573472s 2026-02-25T15:07:59.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.973+0000 7fb826d76640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fb8281a5660 0x7fb8281c5a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45896/0 (socket says 10.20.193.90:45896) 2026-02-25T15:07:59.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.973+0000 7fb826d76640 1 -- 10.20.193.90:0/851889967 learned_addr learned my addr 10.20.193.90:0/851889967 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:59.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 1 -- 10.20.193.90:0/851889967 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3747778053 0 0) 0x7fb8281630d0 con 0x7fb8281a5660 2026-02-25T15:07:59.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 10 monclient(hunting): my global_id is 27568 2026-02-25T15:07:59.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 1 -- 10.20.193.90:0/851889967 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb814003610 con 0x7fb8281a5660 2026-02-25T15:07:59.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 1 -- 10.20.193.90:0/851889967 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1190169145 0 0) 0x7fb828162df0 con 0x7fb82815e2a0 2026-02-25T15:07:59.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 10 monclient(hunting): my global_id is 30939 2026-02-25T15:07:59.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 1 -- 10.20.193.90:0/851889967 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8281630d0 con 0x7fb82815e2a0 2026-02-25T15:07:59.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 1 -- 10.20.193.90:0/851889967 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 653893374 0 0) 0x7fb8281633b0 con 0x7fb828059e10 2026-02-25T15:07:59.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 10 monclient(hunting): my global_id is 28706 2026-02-25T15:07:59.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.975+0000 7fb825573640 1 -- 10.20.193.90:0/851889967 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb828162df0 con 0x7fb828059e10 2026-02-25T15:07:59.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb825573640 1 -- 10.20.193.90:0/851889967 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1087673495 0 0) 0x7fb828162df0 con 0x7fb828059e10 2026-02-25T15:07:59.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb825573640 1 -- 10.20.193.90:0/851889967 >> v1:10.20.193.93:6789/0 conn(0x7fb8281a5660 legacy=0x7fb8281c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb825573640 1 -- 10.20.193.90:0/851889967 >> v1:10.20.193.90:6789/0 conn(0x7fb82815e2a0 legacy=0x7fb82805da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb825573640 10 monclient: _finish_hunting 0 2026-02-25T15:07:59.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb825573640 1 monclient: found mon.noname-c 2026-02-25T15:07:59.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb825573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:07:59.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb825573640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:07:59.987 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb825573640 1 -- 10.20.193.90:0/851889967 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb828163690 con 0x7fb828059e10 2026-02-25T15:07:59.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb825573640 10 monclient: _finish_auth 0 2026-02-25T15:07:59.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb825573640 10 monclient: _check_auth_tickets 2026-02-25T15:07:59.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb825573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:07:59.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb825573640 1 -- 10.20.193.90:0/851889967 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb810002dd0 con 0x7fb828059e10 2026-02-25T15:07:59.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb825573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:59.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb825573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:07:59.988 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb825573640 10 monclient: dump: 2026-02-25T15:07:59.988 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.988 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.988 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.988 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.988 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.989 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.989 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.989 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.989 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.989 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.989 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb82cdc8640 5 monclient: authenticate success, global_id 28706 2026-02-25T15:07:59.989 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.977+0000 7fb82cdc8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:07:59.989 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb825573640 1 -- 10.20.193.90:0/851889967 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb810002fa0 con 0x7fb828059e10 2026-02-25T15:07:59.989 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb825573640 10 monclient: handle_config config(0 keys) 2026-02-25T15:07:59.989 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb825573640 1 -- 10.20.193.90:0/851889967 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb8100033e0 con 0x7fb828059e10 2026-02-25T15:07:59.989 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb825573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:07:59.989 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb825573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:59.990 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb825573640 10 monclient: dump: 2026-02-25T15:07:59.990 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.990 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.990 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.990 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.990 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.990 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.990 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.990 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.990 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.990 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.990 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb82cdc8640 10 monclient: get_monmap_and_config success 2026-02-25T15:07:59.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb82cdc8640 4 set_mon_vals no callback set 2026-02-25T15:07:59.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb82cdc8640 10 monclient: shutdown 2026-02-25T15:07:59.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb82cdc8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:07:59.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb82cdc8640 1 -- 10.20.193.90:0/851889967 >> v1:10.20.193.94:6789/0 conn(0x7fb828059e10 legacy=0x7fb82815ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:07:59.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb82cdc8640 1 -- 10.20.193.90:0/851889967 shutdown_connections 2026-02-25T15:07:59.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb82cdc8640 1 -- 10.20.193.90:0/851889967 >> 10.20.193.90:0/851889967 conn(0x7fb8280b2f60 msgr2=0x7fb8280b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:07:59.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb82cdc8640 1 -- 10.20.193.90:0/851889967 shutdown_connections 2026-02-25T15:07:59.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb82cdc8640 1 -- 10.20.193.90:0/851889967 wait complete. 2026-02-25T15:07:59.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb82cdc8640 10 monclient: build_initial_monmap 2026-02-25T15:07:59.991 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.978+0000 7fb82cdc8640 10 monclient: monmap: 2026-02-25T15:07:59.991 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:07:59.991 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:07:59.992 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:07:59.992 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:07:59.992 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:07:59.992 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:07:59.992 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:07:59.992 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:07:59.992 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:07:59.992 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.992 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 1 librados: starting msgr at 2026-02-25T15:07:59.992 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 1 librados: starting objecter 2026-02-25T15:07:59.992 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:07:59.992 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 1 Processor -- start 2026-02-25T15:07:59.992 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 1 -- start start 2026-02-25T15:07:59.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 1 librados: setting wanted keys 2026-02-25T15:07:59.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 1 librados: calling monclient init 2026-02-25T15:07:59.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 10 monclient: init 2026-02-25T15:07:59.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 10 monclient: _reopen_session rank -1 2026-02-25T15:07:59.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:07:59.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 10 monclient(hunting): picked mon.noname-c con 0x7fb828059e10 addr v1:10.20.193.94:6789/0 2026-02-25T15:07:59.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 10 monclient(hunting): picked mon.noname-b con 0x7fb82815e2a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:07:59.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 10 monclient(hunting): picked mon.noname-a con 0x7fb8281a5660 addr v1:10.20.193.90:6789/0 2026-02-25T15:07:59.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8280adf80 con 0x7fb8281a5660 2026-02-25T15:07:59.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8280ae260 con 0x7fb82815e2a0 2026-02-25T15:07:59.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8280ae540 con 0x7fb828059e10 2026-02-25T15:07:59.993 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 10 monclient(hunting): _renew_subs 2026-02-25T15:07:59.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb82cdc8640 10 monclient(hunting): authenticate will time out at 3369.579548s 2026-02-25T15:07:59.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb826d76640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fb8281a5660 0x7fb828056b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:40084/0 (socket says 10.20.193.90:40084) 2026-02-25T15:07:59.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb826d76640 1 -- 10.20.193.90:0/883210022 learned_addr learned my addr 10.20.193.90:0/883210022 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:07:59.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb80effd640 10 client.?.objecter ms_handle_connect 0x7fb8281a5660 2026-02-25T15:07:59.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.979+0000 7fb80effd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:59.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.980+0000 7fb80effd640 10 client.?.objecter ms_handle_connect 0x7fb82815e2a0 2026-02-25T15:07:59.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.980+0000 7fb80effd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:59.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.980+0000 7fb80effd640 10 client.?.objecter ms_handle_connect 0x7fb828059e10 2026-02-25T15:07:59.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.980+0000 7fb80effd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:07:59.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.980+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2445548604 0 0) 0x7fb8280adf80 con 0x7fb8281a5660 2026-02-25T15:07:59.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.980+0000 7fb80effd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.980+0000 7fb80effd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.994 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.980+0000 7fb80effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.980+0000 7fb80effd640 10 monclient(hunting): my global_id is 30942 2026-02-25T15:07:59.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.980+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7f4003610 con 0x7fb8281a5660 2026-02-25T15:07:59.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.981+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 49774259 0 0) 0x7fb8280ae260 con 0x7fb82815e2a0 2026-02-25T15:07:59.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.981+0000 7fb80effd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.981+0000 7fb80effd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.981+0000 7fb80effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.981+0000 7fb80effd640 10 monclient(hunting): my global_id is 27571 2026-02-25T15:07:59.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.981+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8280adf80 con 0x7fb82815e2a0 2026-02-25T15:07:59.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.981+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3087449007 0 0) 0x7fb8280ae540 con 0x7fb828059e10 2026-02-25T15:07:59.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.981+0000 7fb80effd640 10 monclient(hunting): _negotiate 2026-02-25T15:07:59.995 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.981+0000 7fb80effd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:07:59.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.981+0000 7fb80effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:07:59.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.981+0000 7fb80effd640 10 monclient(hunting): my global_id is 28709 2026-02-25T15:07:59.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.981+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8280ae260 con 0x7fb828059e10 2026-02-25T15:07:59.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1589896845 0 0) 0x7fb7f4003610 con 0x7fb8281a5660 2026-02-25T15:07:59.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb8280ae540 con 0x7fb8281a5660 2026-02-25T15:07:59.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb818003430 con 0x7fb8281a5660 2026-02-25T15:07:59.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:59.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:07:59.996 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 10 monclient(hunting): dump: 2026-02-25T15:07:59.996 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.996 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.996 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.997 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.997 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.997 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.997 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.997 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.997 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.997 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:07:59.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 424401718 0 0) 0x7fb8280adf80 con 0x7fb82815e2a0 2026-02-25T15:07:59.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb7f4003610 con 0x7fb82815e2a0 2026-02-25T15:07:59.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb8100032f0 con 0x7fb82815e2a0 2026-02-25T15:07:59.997 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:59.998 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:07:59.998 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 10 monclient(hunting): dump: 2026-02-25T15:07:59.998 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.998 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.998 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.998 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.998 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.998 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.998 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.998 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:07:59.998 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:07:59.998 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:07:59.998 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 36871646 0 0) 0x7fb8280ae260 con 0x7fb828059e10 2026-02-25T15:07:59.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb8280adf80 con 0x7fb828059e10 2026-02-25T15:07:59.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb81c0030b0 con 0x7fb828059e10 2026-02-25T15:07:59.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:07:59.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:07:59.999 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.983+0000 7fb80effd640 10 monclient(hunting): dump: 2026-02-25T15:07:59.999 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:07:59.999 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:07:59.999 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.999 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:07:59.999 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:07:59.999 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:07:59.999 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:07:59.999 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.000 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.000 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2479472029 0 0) 0x7fb8280ae540 con 0x7fb8281a5660 2026-02-25T15:08:00.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 >> v1:10.20.193.94:6789/0 conn(0x7fb828059e10 legacy=0x7fb8280af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 >> v1:10.20.193.93:6789/0 conn(0x7fb82815e2a0 legacy=0x7fb8281a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb80effd640 10 monclient: _finish_hunting 0 2026-02-25T15:08:00.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb80effd640 1 monclient: found mon.a 2026-02-25T15:08:00.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb80effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:00.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb80effd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:08:00.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8280ae820 con 0x7fb8281a5660 2026-02-25T15:08:00.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb80effd640 10 monclient: _finish_auth 0 2026-02-25T15:08:00.000 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb80effd640 10 monclient: _check_auth_tickets 2026-02-25T15:08:00.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb80effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:00.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb82cdc8640 5 monclient: authenticate success, global_id 30942 2026-02-25T15:08:00.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb82cdc8640 10 monclient: _renew_subs 2026-02-25T15:08:00.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb82cdc8640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:08:00.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb82cdc8640 1 -- 10.20.193.90:0/883210022 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb8280aeb00 con 0x7fb8281a5660 2026-02-25T15:08:00.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb82cdc8640 10 client.30942.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:08:00.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb82cdc8640 10 monclient: _renew_subs 2026-02-25T15:08:00.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb82cdc8640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:08:00.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb82cdc8640 1 -- 10.20.193.90:0/883210022 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb82815fdf0 con 0x7fb8281a5660 2026-02-25T15:08:00.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb82cdc8640 1 librados: init done 2026-02-25T15:08:00.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb82cdc8640 10 client.30942.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:00.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb82cdc8640 15 client.30942.objecter _session_command_op_assign -1 1 2026-02-25T15:08:00.001 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb82cdc8640 20 client.30942.objecter _get_session osd=-1 returning homeless 2026-02-25T15:08:00.002 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb82cdc8640 10 client.30942.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:08:00.002 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb82cdc8640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:08:00.002 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.985+0000 7fb82cdc8640 1 -- 10.20.193.90:0/883210022 --> v1:10.20.193.90:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fb8280b2cb0 con 0x7fb8281a5660 2026-02-25T15:08:00.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.994+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb8280b2cb0 con 0x7fb8281a5660 2026-02-25T15:08:00.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.994+0000 7fb80effd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:00.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.994+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fb818004df0 con 0x7fb8281a5660 2026-02-25T15:08:00.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.994+0000 7fb80effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:00.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.994+0000 7fb80effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:08:00.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.994+0000 7fb824d72640 4 set_mon_vals no callback set 2026-02-25T15:08:00.004 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.994+0000 7fb80effd640 10 monclient: dump: 2026-02-25T15:08:00.005 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.005 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.005 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.005 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.005 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.005 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.005 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.005 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.005 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.005 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.005 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.994+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fb81801e2d0 con 0x7fb8281a5660 2026-02-25T15:08:00.005 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.995+0000 7fb826575640 10 monclient: get_auth_request con 0x7fb7f407a040 auth_method 0 2026-02-25T15:08:00.006 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.995+0000 7fb80effd640 10 client.30942.objecter ms_handle_connect 0x7fb7f407a040 2026-02-25T15:08:00.006 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7fb818095c60 con 0x7fb8281a5660 2026-02-25T15:08:00.006 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 10 client.30942.objecter ms_dispatch2 0x7fb828000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:08:00.006 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 3 client.30942.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:08:00.006 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 3 client.30942.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:08:00.006 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 10 client.30942.objecter checking command 1 2026-02-25T15:08:00.006 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 20 client.30942.objecter _get_session s=0x7fb7f40a42f0 osd=7 3 2026-02-25T15:08:00.006 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 20 client.30942.objecter put_session s=0x7fb7f40a42f0 osd=7 3 2026-02-25T15:08:00.006 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 15 client.30942.objecter _session_command_op_remove -1 1 2026-02-25T15:08:00.006 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 20 client.30942.objecter _get_session s=0x7fb7f40a42f0 osd=7 3 2026-02-25T15:08:00.006 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 20 client.30942.objecter get_session s=0x7fb7f40a42f0 osd=7 3 2026-02-25T15:08:00.006 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 15 client.30942.objecter _session_command_op_assign 7 1 2026-02-25T15:08:00.007 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 20 client.30942.objecter put_session s=0x7fb7f40a42f0 osd=7 4 2026-02-25T15:08:00.007 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 10 client.30942.objecter _send_command 1 2026-02-25T15:08:00.007 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 --> v1:10.20.193.189:6809/1708410075 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb7f40c6600 con 0x7fb7f40a49a0 2026-02-25T15:08:00.007 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 20 client.30942.objecter dump_active .. 0 homeless 2026-02-25T15:08:00.007 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_get_version_reply(handle=1 version=338) ==== 24+0+0 (unknown 1001891302 0 0) 0x7fb81805ec90 con 0x7fb8281a5660 2026-02-25T15:08:00.007 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb80effd640 10 monclient: handle_get_version_reply finishing 1 version 338 2026-02-25T15:08:00.007 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.996+0000 7fb825d74640 10 monclient: get_auth_request con 0x7fb7f40a49a0 auth_method 0 2026-02-25T15:08:00.007 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:07:59.997+0000 7fb80effd640 10 client.30942.objecter ms_handle_connect 0x7fb7f40a49a0 2026-02-25T15:08:00.008 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.007+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== osd.7 v1:10.20.193.189:6809/1708410075 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7fb7f40c6600 con 0x7fb7f40a49a0 2026-02-25T15:08:00.008 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.007+0000 7fb80effd640 10 client.30942.objecter ms_dispatch2 0x7fb828000b90 command_reply(tid 1: 0 ) 2026-02-25T15:08:00.009 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.007+0000 7fb80effd640 10 client.30942.objecter _finish_command 1 = system:0 2026-02-25T15:08:00.009 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.007+0000 7fb80effd640 20 client.30942.objecter put_session s=0x7fb7f40a42f0 osd=7 4 2026-02-25T15:08:00.009 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.007+0000 7fb80effd640 15 client.30942.objecter _session_command_op_remove 7 1 2026-02-25T15:08:00.016 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.014+0000 7fb82cdc8640 10 client.30942.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:08:00.016 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.014+0000 7fb82cdc8640 15 client.30942.objecter _session_command_op_assign -1 2 2026-02-25T15:08:00.016 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.014+0000 7fb82cdc8640 20 client.30942.objecter _get_session s=0x7fb7f40a42f0 osd=7 3 2026-02-25T15:08:00.016 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.014+0000 7fb82cdc8640 20 client.30942.objecter put_session s=0x7fb7f40a42f0 osd=7 3 2026-02-25T15:08:00.016 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.014+0000 7fb82cdc8640 20 client.30942.objecter _get_session s=0x7fb7f40a42f0 osd=7 3 2026-02-25T15:08:00.016 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.014+0000 7fb82cdc8640 15 client.30942.objecter _session_command_op_remove -1 2 2026-02-25T15:08:00.016 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.014+0000 7fb82cdc8640 20 client.30942.objecter get_session s=0x7fb7f40a42f0 osd=7 3 2026-02-25T15:08:00.017 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.014+0000 7fb82cdc8640 15 client.30942.objecter _session_command_op_assign 7 2 2026-02-25T15:08:00.017 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.014+0000 7fb82cdc8640 20 client.30942.objecter put_session s=0x7fb7f40a42f0 osd=7 4 2026-02-25T15:08:00.017 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.014+0000 7fb82cdc8640 10 client.30942.objecter _send_command 2 2026-02-25T15:08:00.017 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.014+0000 7fb82cdc8640 1 -- 10.20.193.90:0/883210022 --> v1:10.20.193.189:6809/1708410075 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb8280a7b20 con 0x7fb7f40a49a0 2026-02-25T15:08:00.024 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb80effd640 1 -- 10.20.193.90:0/883210022 <== osd.7 v1:10.20.193.189:6809/1708410075 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7fb8280a7b20 con 0x7fb7f40a49a0 2026-02-25T15:08:00.025 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb80effd640 10 client.30942.objecter ms_dispatch2 0x7fb828000b90 command_reply(tid 2: 0 ) 2026-02-25T15:08:00.025 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb80effd640 10 client.30942.objecter _finish_command 2 = system:0 2026-02-25T15:08:00.025 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb80effd640 20 client.30942.objecter put_session s=0x7fb7f40a42f0 osd=7 4 2026-02-25T15:08:00.025 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb80effd640 15 client.30942.objecter _session_command_op_remove 7 2 2026-02-25T15:08:00.026 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:08:00.026 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-25T15:08:00.026 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:08:00.029 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb82cdc8640 10 librados: watch_flush enter 2026-02-25T15:08:00.029 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb82cdc8640 10 librados: watch_flush exit 2026-02-25T15:08:00.029 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb82cdc8640 10 client.30942.objecter close_session for osd.7 2026-02-25T15:08:00.030 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb82cdc8640 1 -- 10.20.193.90:0/883210022 >> v1:10.20.193.189:6809/1708410075 conn(0x7fb7f40a49a0 legacy=0x7fb7f40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.030 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb82cdc8640 20 client.30942.objecter put_session s=0x7fb7f40a42f0 osd=7 1 2026-02-25T15:08:00.030 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb82cdc8640 20 client.30942.objecter shutdown clearing up homeless session... 2026-02-25T15:08:00.030 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb82cdc8640 10 client.30942.objecter successfully canceled tick 2026-02-25T15:08:00.030 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb82cdc8640 1 -- 10.20.193.90:0/883210022 >> v1:10.20.193.189:6800/3647375982 conn(0x7fb7f407a040 legacy=0x7fb7f409a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.031 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb82cdc8640 10 monclient: shutdown 2026-02-25T15:08:00.031 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb82cdc8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:00.031 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb82cdc8640 1 -- 10.20.193.90:0/883210022 >> v1:10.20.193.90:6789/0 conn(0x7fb8281a5660 legacy=0x7fb828056b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.031 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb826d76640 1 -- 10.20.193.90:0/883210022 reap_dead start 2026-02-25T15:08:00.031 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb82cdc8640 1 -- 10.20.193.90:0/883210022 shutdown_connections 2026-02-25T15:08:00.031 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.022+0000 7fb82cdc8640 1 -- 10.20.193.90:0/883210022 >> 10.20.193.90:0/883210022 conn(0x7fb8280b2f60 msgr2=0x7fb82815bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:00.032 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.023+0000 7fb82cdc8640 1 -- 10.20.193.90:0/883210022 shutdown_connections 2026-02-25T15:08:00.032 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.023+0000 7fb82cdc8640 1 -- 10.20.193.90:0/883210022 wait complete. 2026-02-25T15:08:00.032 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.023+0000 7fb82cdc8640 1 librados: shutdown 2026-02-25T15:08:00.034 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2026-02-25T15:08:00.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.101+0000 7ff91e591640 10 monclient: get_monmap_and_config 2026-02-25T15:08:00.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.101+0000 7ff91e591640 10 monclient: build_initial_monmap 2026-02-25T15:08:00.103 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.101+0000 7ff91e591640 10 monclient: monmap: 2026-02-25T15:08:00.103 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:00.103 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.103 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:08:00.102272+0000 2026-02-25T15:08:00.103 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:08:00.102272+0000 2026-02-25T15:08:00.103 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:00.103 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.103 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:00.103 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:00.103 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:00.103 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.102+0000 7ff91e591640 1 Processor -- start 2026-02-25T15:08:00.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.102+0000 7ff91e591640 1 -- start start 2026-02-25T15:08:00.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.102+0000 7ff91e591640 10 monclient: init 2026-02-25T15:08:00.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.102+0000 7ff91e591640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:00.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.102+0000 7ff91e591640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:08:00.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.102+0000 7ff91e591640 10 monclient(hunting): picked mon.noname-a con 0x7ff918157780 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:00.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.102+0000 7ff91e591640 10 monclient(hunting): picked mon.noname-c con 0x7ff918157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:00.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.102+0000 7ff91e591640 10 monclient(hunting): picked mon.noname-b con 0x7ff918164d00 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:00.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.102+0000 7ff91e591640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff91805de70 con 0x7ff918157780 2026-02-25T15:08:00.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.102+0000 7ff91e591640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff918055b20 con 0x7ff918164d00 2026-02-25T15:08:00.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.102+0000 7ff91e591640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff918055e00 con 0x7ff918157b50 2026-02-25T15:08:00.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.102+0000 7ff91e591640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:00.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.102+0000 7ff91e591640 10 monclient(hunting): authenticate will time out at 3369.702624s 2026-02-25T15:08:00.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.102+0000 7ff917fff640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7ff918157780 0x7ff918056ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:40100/0 (socket says 10.20.193.90:40100) 2026-02-25T15:08:00.104 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.102+0000 7ff917fff640 1 -- 10.20.193.90:0/632829356 learned_addr learned my addr 10.20.193.90:0/632829356 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:00.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 1 -- 10.20.193.90:0/632829356 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1597037988 0 0) 0x7ff91805de70 con 0x7ff918157780 2026-02-25T15:08:00.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 10 monclient(hunting): my global_id is 30945 2026-02-25T15:08:00.105 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 1 -- 10.20.193.90:0/632829356 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8fc003610 con 0x7ff918157780 2026-02-25T15:08:00.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 1 -- 10.20.193.90:0/632829356 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4224523 0 0) 0x7ff918055b20 con 0x7ff918164d00 2026-02-25T15:08:00.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 10 monclient(hunting): my global_id is 27574 2026-02-25T15:08:00.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 1 -- 10.20.193.90:0/632829356 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff91805de70 con 0x7ff918164d00 2026-02-25T15:08:00.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 1 -- 10.20.193.90:0/632829356 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 312731052 0 0) 0x7ff918055e00 con 0x7ff918157b50 2026-02-25T15:08:00.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 10 monclient(hunting): my global_id is 28712 2026-02-25T15:08:00.106 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.104+0000 7ff916ffd640 1 -- 10.20.193.90:0/632829356 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff918055b20 con 0x7ff918157b50 2026-02-25T15:08:00.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff916ffd640 1 -- 10.20.193.90:0/632829356 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 5632877 0 0) 0x7ff8fc003610 con 0x7ff918157780 2026-02-25T15:08:00.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff916ffd640 1 -- 10.20.193.90:0/632829356 >> v1:10.20.193.94:6789/0 conn(0x7ff918157b50 legacy=0x7ff9181645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff916ffd640 1 -- 10.20.193.90:0/632829356 >> v1:10.20.193.93:6789/0 conn(0x7ff918164d00 legacy=0x7ff9181c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff916ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:08:00.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff916ffd640 1 monclient: found mon.noname-a 2026-02-25T15:08:00.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff916ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:00.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff916ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:08:00.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff916ffd640 1 -- 10.20.193.90:0/632829356 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9180560e0 con 0x7ff918157780 2026-02-25T15:08:00.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff916ffd640 10 monclient: _finish_auth 0 2026-02-25T15:08:00.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff916ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:08:00.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff916ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:00.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff916ffd640 1 -- 10.20.193.90:0/632829356 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff904002df0 con 0x7ff918157780 2026-02-25T15:08:00.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff916ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:00.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff916ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:08:00.107 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff916ffd640 10 monclient: dump: 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff91e591640 5 monclient: authenticate success, global_id 30945 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.106+0000 7ff91e591640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff916ffd640 1 -- 10.20.193.90:0/632829356 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff904002fc0 con 0x7ff918157780 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff916ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff916ffd640 1 -- 10.20.193.90:0/632829356 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff904003400 con 0x7ff918157780 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff916ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff916ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff916ffd640 10 monclient: dump: 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.108 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff91e591640 10 monclient: get_monmap_and_config success 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff91e591640 4 set_mon_vals no callback set 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff91e591640 10 monclient: shutdown 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff91e591640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff91e591640 1 -- 10.20.193.90:0/632829356 >> v1:10.20.193.90:6789/0 conn(0x7ff918157780 legacy=0x7ff918056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff91e591640 1 -- 10.20.193.90:0/632829356 shutdown_connections 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff91e591640 1 -- 10.20.193.90:0/632829356 >> 10.20.193.90:0/632829356 conn(0x7ff9180b2ef0 msgr2=0x7ff9180b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff91e591640 1 -- 10.20.193.90:0/632829356 shutdown_connections 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff91e591640 1 -- 10.20.193.90:0/632829356 wait complete. 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff91e591640 10 monclient: build_initial_monmap 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.107+0000 7ff91e591640 10 monclient: monmap: 2026-02-25T15:08:00.109 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 1 librados: starting msgr at 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 1 librados: starting objecter 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 1 Processor -- start 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 1 -- start start 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 1 librados: setting wanted keys 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 1 librados: calling monclient init 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 10 monclient: init 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 10 monclient(hunting): picked mon.noname-a con 0x7ff918157780 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 10 monclient(hunting): picked mon.noname-c con 0x7ff918157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 10 monclient(hunting): picked mon.noname-b con 0x7ff918164d00 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff918130c10 con 0x7ff918157780 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff918144430 con 0x7ff918164d00 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff918144710 con 0x7ff918157b50 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:00.110 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91e591640 10 monclient(hunting): authenticate will time out at 3369.708595s 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91cb07640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7ff918164d00 0x7ff9181326b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45924/0 (socket says 10.20.193.90:45924) 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.108+0000 7ff91cb07640 1 -- 10.20.193.90:0/3558012706 learned_addr learned my addr 10.20.193.90:0/3558012706 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.109+0000 7ff914ff9640 10 client.?.objecter ms_handle_connect 0x7ff918157b50 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.109+0000 7ff914ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.109+0000 7ff914ff9640 10 client.?.objecter ms_handle_connect 0x7ff918164d00 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.109+0000 7ff914ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.109+0000 7ff914ff9640 10 client.?.objecter ms_handle_connect 0x7ff918157780 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.109+0000 7ff914ff9640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3877733450 0 0) 0x7ff918144430 con 0x7ff918164d00 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 10 monclient(hunting): my global_id is 27577 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8e8003610 con 0x7ff918164d00 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 58718569 0 0) 0x7ff918144710 con 0x7ff918157b50 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.111 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 10 monclient(hunting): my global_id is 28715 2026-02-25T15:08:00.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff918144430 con 0x7ff918157b50 2026-02-25T15:08:00.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2275488858 0 0) 0x7ff918130c10 con 0x7ff918157780 2026-02-25T15:08:00.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 10 monclient(hunting): my global_id is 30948 2026-02-25T15:08:00.112 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.110+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff918144710 con 0x7ff918157780 2026-02-25T15:08:00.113 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.112+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 991982810 0 0) 0x7ff918144430 con 0x7ff918157b50 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.112+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff918130c10 con 0x7ff918157b50 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.112+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff908003060 con 0x7ff918157b50 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.112+0000 7ff914ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.112+0000 7ff914ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.112+0000 7ff914ff9640 10 monclient(hunting): dump: 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.112+0000 7ff914ff9640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.113+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3932503308 0 0) 0x7ff8e8003610 con 0x7ff918164d00 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.113+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff918144430 con 0x7ff918164d00 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.113+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff90c0033f0 con 0x7ff918164d00 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.113+0000 7ff914ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.113+0000 7ff914ff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.113+0000 7ff914ff9640 10 monclient(hunting): dump: 2026-02-25T15:08:00.114 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.113+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 330129497 0 0) 0x7ff918144710 con 0x7ff918157780 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.113+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff8e8003610 con 0x7ff918157780 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.113+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff904003310 con 0x7ff918157780 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.113+0000 7ff914ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.113+0000 7ff914ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.113+0000 7ff914ff9640 10 monclient(hunting): dump: 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.115 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1433889034 0 0) 0x7ff918144430 con 0x7ff918164d00 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 >> v1:10.20.193.94:6789/0 conn(0x7ff918157b50 legacy=0x7ff9180ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 >> v1:10.20.193.90:6789/0 conn(0x7ff918157780 legacy=0x7ff918131680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff914ff9640 10 monclient: _finish_hunting 0 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff914ff9640 1 monclient: found mon.b 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff914ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff914ff9640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9181449f0 con 0x7ff918164d00 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff914ff9640 10 monclient: _finish_auth 0 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff914ff9640 10 monclient: _check_auth_tickets 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff914ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff91e591640 5 monclient: authenticate success, global_id 27577 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff91e591640 10 monclient: _renew_subs 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff91e591640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff91e591640 1 -- 10.20.193.90:0/3558012706 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff918130e80 con 0x7ff918164d00 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff91e591640 10 client.27577.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff91e591640 10 monclient: _renew_subs 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff91e591640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff91e591640 1 -- 10.20.193.90:0/3558012706 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff918144cd0 con 0x7ff918164d00 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.114+0000 7ff91e591640 1 librados: init done 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.115+0000 7ff91e591640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:00.116 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.115+0000 7ff91e591640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:00.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.115+0000 7ff91e591640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.117 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.115+0000 7ff91e591640 1 -- 10.20.193.90:0/3558012706 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff91805e250 con 0x7ff918164d00 2026-02-25T15:08:00.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.209+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff90c003a80 con 0x7ff918164d00 2026-02-25T15:08:00.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.209+0000 7ff914ff9640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:00.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.209+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff90c003e50 con 0x7ff918164d00 2026-02-25T15:08:00.211 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.209+0000 7ff914ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:00.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.209+0000 7ff914ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:08:00.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.209+0000 7ff9167fc640 4 set_mon_vals no callback set 2026-02-25T15:08:00.212 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.209+0000 7ff914ff9640 10 monclient: dump: 2026-02-25T15:08:00.212 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.212 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.212 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.212 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.212 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.212 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.212 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.212 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.212 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.213 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.210+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7ff90c003e50 con 0x7ff918164d00 2026-02-25T15:08:00.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.210+0000 7ff917fff640 10 monclient: get_auth_request con 0x7ff8e807a190 auth_method 0 2026-02-25T15:08:00.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.211+0000 7ff914ff9640 10 client.27577.objecter ms_handle_connect 0x7ff8e807a190 2026-02-25T15:08:00.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.211+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7ff90c095d40 con 0x7ff918164d00 2026-02-25T15:08:00.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.211+0000 7ff914ff9640 10 client.27577.objecter ms_dispatch2 0x7ff918000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:08:00.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.211+0000 7ff914ff9640 3 client.27577.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:08:00.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.211+0000 7ff914ff9640 3 client.27577.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:08:00.214 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.212+0000 7ff914ff9640 20 client.27577.objecter dump_active .. 0 homeless 2026-02-25T15:08:00.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.213+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7ff91805e250 con 0x7ff918164d00 2026-02-25T15:08:00.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.213+0000 7ff914ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:00.215 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.213+0000 7ff914ff9640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:08:00.321 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.320+0000 7ff91e591640 1 -- 10.20.193.90:0/3558012706 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7ff9180b3910 con 0x7ff8e807a190 2026-02-25T15:08:00.322 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.320+0000 7ff914ff9640 1 -- 10.20.193.90:0/3558012706 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (unknown 1239724482 0 0) 0x7ff9180b3910 con 0x7ff8e807a190 2026-02-25T15:08:00.322 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 7 to deep-scrub 2026-02-25T15:08:00.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.321+0000 7ff91e591640 10 librados: watch_flush enter 2026-02-25T15:08:00.323 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.321+0000 7ff91e591640 10 librados: watch_flush exit 2026-02-25T15:08:00.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.321+0000 7ff91e591640 20 client.27577.objecter shutdown clearing up homeless session... 2026-02-25T15:08:00.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.321+0000 7ff91e591640 10 client.27577.objecter successfully canceled tick 2026-02-25T15:08:00.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.321+0000 7ff91e591640 1 -- 10.20.193.90:0/3558012706 >> v1:10.20.193.189:6800/3647375982 conn(0x7ff8e807a190 legacy=0x7ff8e809a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.321+0000 7ff91e591640 10 monclient: shutdown 2026-02-25T15:08:00.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.321+0000 7ff91e591640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:00.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.321+0000 7ff91e591640 1 -- 10.20.193.90:0/3558012706 >> v1:10.20.193.93:6789/0 conn(0x7ff918164d00 legacy=0x7ff9181326b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.321+0000 7ff91e591640 1 -- 10.20.193.90:0/3558012706 shutdown_connections 2026-02-25T15:08:00.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.321+0000 7ff91e591640 1 -- 10.20.193.90:0/3558012706 >> 10.20.193.90:0/3558012706 conn(0x7ff9180b2ef0 msgr2=0x7ff9180ad970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:00.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.321+0000 7ff91e591640 1 -- 10.20.193.90:0/3558012706 shutdown_connections 2026-02-25T15:08:00.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.321+0000 7ff91e591640 1 -- 10.20.193.90:0/3558012706 wait complete. 2026-02-25T15:08:00.324 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.321+0000 7ff91e591640 1 librados: shutdown 2026-02-25T15:08:00.332 INFO:tasks.ceph:Scrubbing osd.11 2026-02-25T15:08:00.332 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:08:00.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.401+0000 7fd42f832640 10 monclient: get_monmap_and_config 2026-02-25T15:08:00.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.401+0000 7fd42f832640 10 monclient: build_initial_monmap 2026-02-25T15:08:00.403 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.401+0000 7fd42f832640 10 monclient: monmap: 2026-02-25T15:08:00.403 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:00.404 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.404 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:08:00.401752+0000 2026-02-25T15:08:00.404 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:08:00.401752+0000 2026-02-25T15:08:00.404 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:00.404 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.404 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:00.404 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:00.404 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:00.405 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.401+0000 7fd42f832640 1 Processor -- start 2026-02-25T15:08:00.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.401+0000 7fd42f832640 1 -- start start 2026-02-25T15:08:00.405 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.401+0000 7fd42f832640 10 monclient: init 2026-02-25T15:08:00.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.402+0000 7fd42f832640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:00.406 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.402+0000 7fd42f832640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:08:00.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.402+0000 7fd42f832640 10 monclient(hunting): picked mon.noname-a con 0x7fd42815e2a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:00.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.402+0000 7fd42f832640 10 monclient(hunting): picked mon.noname-b con 0x7fd428059e10 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:00.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.402+0000 7fd42f832640 10 monclient(hunting): picked mon.noname-c con 0x7fd4281a5660 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:00.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.402+0000 7fd42f832640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd428162df0 con 0x7fd42815e2a0 2026-02-25T15:08:00.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.402+0000 7fd42f832640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4281630d0 con 0x7fd428059e10 2026-02-25T15:08:00.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.402+0000 7fd42f832640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4281633b0 con 0x7fd4281a5660 2026-02-25T15:08:00.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.402+0000 7fd42f832640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:00.407 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.402+0000 7fd42f832640 10 monclient(hunting): authenticate will time out at 3370.002122s 2026-02-25T15:08:00.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.409+0000 7fd42cda6640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fd428059e10 0x7fd42815ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45926/0 (socket says 10.20.193.90:45926) 2026-02-25T15:08:00.411 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.409+0000 7fd42cda6640 1 -- 10.20.193.90:0/4029621013 learned_addr learned my addr 10.20.193.90:0/4029621013 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:00.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.411+0000 7fd417fff640 1 -- 10.20.193.90:0/4029621013 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2944392413 0 0) 0x7fd428162df0 con 0x7fd42815e2a0 2026-02-25T15:08:00.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.411+0000 7fd417fff640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.413 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.411+0000 7fd417fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.411+0000 7fd417fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.411+0000 7fd417fff640 10 monclient(hunting): my global_id is 30951 2026-02-25T15:08:00.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.411+0000 7fd417fff640 1 -- 10.20.193.90:0/4029621013 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd410003610 con 0x7fd42815e2a0 2026-02-25T15:08:00.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.411+0000 7fd417fff640 1 -- 10.20.193.90:0/4029621013 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2024280200 0 0) 0x7fd4281630d0 con 0x7fd428059e10 2026-02-25T15:08:00.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.411+0000 7fd417fff640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.411+0000 7fd417fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.411+0000 7fd417fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.411+0000 7fd417fff640 10 monclient(hunting): my global_id is 27580 2026-02-25T15:08:00.414 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.411+0000 7fd417fff640 1 -- 10.20.193.90:0/4029621013 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd428162df0 con 0x7fd428059e10 2026-02-25T15:08:00.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd417fff640 1 -- 10.20.193.90:0/4029621013 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3718104799 0 0) 0x7fd428162df0 con 0x7fd428059e10 2026-02-25T15:08:00.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd417fff640 1 -- 10.20.193.90:0/4029621013 >> v1:10.20.193.94:6789/0 conn(0x7fd4281a5660 legacy=0x7fd4281c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd417fff640 1 -- 10.20.193.90:0/4029621013 >> v1:10.20.193.90:6789/0 conn(0x7fd42815e2a0 legacy=0x7fd42805da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.419 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd417fff640 10 monclient: _finish_hunting 0 2026-02-25T15:08:00.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd417fff640 1 monclient: found mon.noname-b 2026-02-25T15:08:00.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd417fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:00.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd417fff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd417fff640 1 -- 10.20.193.90:0/4029621013 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd428163690 con 0x7fd428059e10 2026-02-25T15:08:00.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd417fff640 10 monclient: _finish_auth 0 2026-02-25T15:08:00.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd417fff640 10 monclient: _check_auth_tickets 2026-02-25T15:08:00.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd417fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:00.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd417fff640 1 -- 10.20.193.90:0/4029621013 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd41c002df0 con 0x7fd428059e10 2026-02-25T15:08:00.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd417fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:00.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd417fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:08:00.420 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd417fff640 10 monclient: dump: 2026-02-25T15:08:00.420 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.421 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.421 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.421 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.421 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.421 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.421 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.421 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.421 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.421 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd42f832640 5 monclient: authenticate success, global_id 27580 2026-02-25T15:08:00.421 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.413+0000 7fd42f832640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:08:00.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.414+0000 7fd417fff640 1 -- 10.20.193.90:0/4029621013 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd41c003000 con 0x7fd428059e10 2026-02-25T15:08:00.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.414+0000 7fd417fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:00.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.414+0000 7fd417fff640 1 -- 10.20.193.90:0/4029621013 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd41c003400 con 0x7fd428059e10 2026-02-25T15:08:00.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.414+0000 7fd417fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:00.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.414+0000 7fd417fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:08:00.422 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.414+0000 7fd417fff640 10 monclient: dump: 2026-02-25T15:08:00.422 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.422 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.422 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.422 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.422 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.423 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.423 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.423 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.423 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.423 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.414+0000 7fd42f832640 10 monclient: get_monmap_and_config success 2026-02-25T15:08:00.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.414+0000 7fd42f832640 4 set_mon_vals no callback set 2026-02-25T15:08:00.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.414+0000 7fd42f832640 10 monclient: shutdown 2026-02-25T15:08:00.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.414+0000 7fd42f832640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:00.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.414+0000 7fd42f832640 1 -- 10.20.193.90:0/4029621013 >> v1:10.20.193.93:6789/0 conn(0x7fd428059e10 legacy=0x7fd42815ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.414+0000 7fd42f832640 1 -- 10.20.193.90:0/4029621013 shutdown_connections 2026-02-25T15:08:00.423 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.414+0000 7fd42f832640 1 -- 10.20.193.90:0/4029621013 >> 10.20.193.90:0/4029621013 conn(0x7fd4280b2f60 msgr2=0x7fd4280b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:00.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.415+0000 7fd42f832640 1 -- 10.20.193.90:0/4029621013 shutdown_connections 2026-02-25T15:08:00.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.415+0000 7fd42f832640 1 -- 10.20.193.90:0/4029621013 wait complete. 2026-02-25T15:08:00.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.415+0000 7fd42f832640 10 monclient: build_initial_monmap 2026-02-25T15:08:00.424 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.415+0000 7fd42f832640 10 monclient: monmap: 2026-02-25T15:08:00.424 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:00.424 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:08:00.424 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:08:00.424 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:08:00.424 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:00.424 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:08:00.424 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:00.425 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:00.425 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:00.425 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.415+0000 7fd42f832640 1 librados: starting msgr at 2026-02-25T15:08:00.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.415+0000 7fd42f832640 1 librados: starting objecter 2026-02-25T15:08:00.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.415+0000 7fd42f832640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:08:00.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.415+0000 7fd42f832640 1 Processor -- start 2026-02-25T15:08:00.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.415+0000 7fd42f832640 1 -- start start 2026-02-25T15:08:00.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.415+0000 7fd42f832640 1 librados: setting wanted keys 2026-02-25T15:08:00.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.415+0000 7fd42f832640 1 librados: calling monclient init 2026-02-25T15:08:00.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.415+0000 7fd42f832640 10 monclient: init 2026-02-25T15:08:00.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd42f832640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:00.425 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd42f832640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:08:00.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd42f832640 10 monclient(hunting): picked mon.noname-b con 0x7fd428059e10 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:00.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd42f832640 10 monclient(hunting): picked mon.noname-c con 0x7fd42815e2a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:00.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd42f832640 10 monclient(hunting): picked mon.noname-a con 0x7fd4281a5660 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:00.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd42f832640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd42812abb0 con 0x7fd4281a5660 2026-02-25T15:08:00.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd42f832640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4280a79e0 con 0x7fd428059e10 2026-02-25T15:08:00.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd42f832640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4280a7cc0 con 0x7fd42815e2a0 2026-02-25T15:08:00.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd42f832640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:00.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd42f832640 10 monclient(hunting): authenticate will time out at 3370.016194s 2026-02-25T15:08:00.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd42dda8640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fd4281a5660 0x7fd428056a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:40116/0 (socket says 10.20.193.90:40116) 2026-02-25T15:08:00.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd42dda8640 1 -- 10.20.193.90:0/2878767398 learned_addr learned my addr 10.20.193.90:0/2878767398 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:00.426 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd415ffb640 10 client.?.objecter ms_handle_connect 0x7fd4281a5660 2026-02-25T15:08:00.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd415ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:00.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd415ffb640 10 client.?.objecter ms_handle_connect 0x7fd428059e10 2026-02-25T15:08:00.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd415ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:00.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd415ffb640 10 client.?.objecter ms_handle_connect 0x7fd42815e2a0 2026-02-25T15:08:00.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.416+0000 7fd415ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:00.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1180171017 0 0) 0x7fd42812abb0 con 0x7fd4281a5660 2026-02-25T15:08:00.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 10 monclient(hunting): my global_id is 30954 2026-02-25T15:08:00.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3f8003610 con 0x7fd4281a5660 2026-02-25T15:08:00.427 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1091151980 0 0) 0x7fd4280a79e0 con 0x7fd428059e10 2026-02-25T15:08:00.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 10 monclient(hunting): my global_id is 27583 2026-02-25T15:08:00.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd42812abb0 con 0x7fd428059e10 2026-02-25T15:08:00.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2355932757 0 0) 0x7fd4280a7cc0 con 0x7fd42815e2a0 2026-02-25T15:08:00.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 10 monclient(hunting): my global_id is 28721 2026-02-25T15:08:00.428 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.417+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4280a79e0 con 0x7fd42815e2a0 2026-02-25T15:08:00.431 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3501663583 0 0) 0x7fd42812abb0 con 0x7fd428059e10 2026-02-25T15:08:00.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd4280a7cc0 con 0x7fd428059e10 2026-02-25T15:08:00.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 47232128 0 0) 0x7fd3f8003610 con 0x7fd4281a5660 2026-02-25T15:08:00.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd42812abb0 con 0x7fd4281a5660 2026-02-25T15:08:00.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd424003400 con 0x7fd4281a5660 2026-02-25T15:08:00.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:00.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:08:00.432 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 10 monclient(hunting): dump: 2026-02-25T15:08:00.432 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.432 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.432 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.432 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.432 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.433 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.433 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.433 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.433 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.433 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:08:00.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd4180031d0 con 0x7fd428059e10 2026-02-25T15:08:00.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:00.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:08:00.433 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 10 monclient(hunting): dump: 2026-02-25T15:08:00.433 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.433 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.433 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.434 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.434 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.434 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.434 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.434 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.434 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.434 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3376638150 0 0) 0x7fd4280a79e0 con 0x7fd42815e2a0 2026-02-25T15:08:00.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd3f8003610 con 0x7fd42815e2a0 2026-02-25T15:08:00.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd41c003350 con 0x7fd42815e2a0 2026-02-25T15:08:00.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:00.434 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:08:00.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.420+0000 7fd415ffb640 10 monclient(hunting): dump: 2026-02-25T15:08:00.435 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.435 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.435 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.435 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.435 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.435 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.435 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.435 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.435 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.435 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1748424310 0 0) 0x7fd4280a7cc0 con 0x7fd428059e10 2026-02-25T15:08:00.435 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 >> v1:10.20.193.94:6789/0 conn(0x7fd42815e2a0 legacy=0x7fd428190550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 >> v1:10.20.193.90:6789/0 conn(0x7fd4281a5660 legacy=0x7fd428056a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd415ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:08:00.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd415ffb640 1 monclient: found mon.b 2026-02-25T15:08:00.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd415ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:00.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd415ffb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4280a7fa0 con 0x7fd428059e10 2026-02-25T15:08:00.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd415ffb640 10 monclient: _finish_auth 0 2026-02-25T15:08:00.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd415ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:08:00.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd415ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:00.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd42f832640 5 monclient: authenticate success, global_id 27583 2026-02-25T15:08:00.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd42f832640 10 monclient: _renew_subs 2026-02-25T15:08:00.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd42f832640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd42f832640 1 -- 10.20.193.90:0/2878767398 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd4280a8280 con 0x7fd428059e10 2026-02-25T15:08:00.436 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd42f832640 10 client.27583.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:08:00.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd42f832640 10 monclient: _renew_subs 2026-02-25T15:08:00.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd42f832640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd42f832640 1 -- 10.20.193.90:0/2878767398 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd428190c60 con 0x7fd428059e10 2026-02-25T15:08:00.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.421+0000 7fd42f832640 1 librados: init done 2026-02-25T15:08:00.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.422+0000 7fd42f832640 10 client.27583.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:00.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.422+0000 7fd42f832640 15 client.27583.objecter _session_command_op_assign -1 1 2026-02-25T15:08:00.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.422+0000 7fd42f832640 20 client.27583.objecter _get_session osd=-1 returning homeless 2026-02-25T15:08:00.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.422+0000 7fd42f832640 10 client.27583.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:08:00.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.422+0000 7fd42f832640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.422+0000 7fd42f832640 1 -- 10.20.193.90:0/2878767398 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fd428190f40 con 0x7fd428059e10 2026-02-25T15:08:00.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.426+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd428190f40 con 0x7fd428059e10 2026-02-25T15:08:00.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.426+0000 7fd415ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:00.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.426+0000 7fd4177fe640 4 set_mon_vals no callback set 2026-02-25T15:08:00.437 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.429+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fd4180038a0 con 0x7fd428059e10 2026-02-25T15:08:00.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.429+0000 7fd415ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:00.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.429+0000 7fd415ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:08:00.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.429+0000 7fd415ffb640 10 monclient: dump: 2026-02-25T15:08:00.438 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.438 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.438 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.438 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.438 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.438 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.438 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.438 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.438 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.438 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.438 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.430+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fd41801cd80 con 0x7fd428059e10 2026-02-25T15:08:00.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd42cda6640 10 monclient: get_auth_request con 0x7fd3f807a090 auth_method 0 2026-02-25T15:08:00.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 10 client.27583.objecter ms_handle_connect 0x7fd3f807a090 2026-02-25T15:08:00.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7fd418059990 con 0x7fd428059e10 2026-02-25T15:08:00.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 10 client.27583.objecter ms_dispatch2 0x7fd428000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:08:00.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 3 client.27583.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:08:00.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 3 client.27583.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:08:00.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 10 client.27583.objecter checking command 1 2026-02-25T15:08:00.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 20 client.27583.objecter _get_session s=0x7fd3f80a43a0 osd=11 3 2026-02-25T15:08:00.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 20 client.27583.objecter put_session s=0x7fd3f80a43a0 osd=11 3 2026-02-25T15:08:00.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 15 client.27583.objecter _session_command_op_remove -1 1 2026-02-25T15:08:00.440 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 20 client.27583.objecter _get_session s=0x7fd3f80a43a0 osd=11 3 2026-02-25T15:08:00.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 20 client.27583.objecter get_session s=0x7fd3f80a43a0 osd=11 3 2026-02-25T15:08:00.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 15 client.27583.objecter _session_command_op_assign 11 1 2026-02-25T15:08:00.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 20 client.27583.objecter put_session s=0x7fd3f80a43a0 osd=11 4 2026-02-25T15:08:00.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 10 client.27583.objecter _send_command 1 2026-02-25T15:08:00.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 --> v1:10.20.193.189:6801/4031575111 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3f80c66b0 con 0x7fd3f80a4a50 2026-02-25T15:08:00.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.431+0000 7fd415ffb640 20 client.27583.objecter dump_active .. 0 homeless 2026-02-25T15:08:00.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.432+0000 7fd42dda8640 10 monclient: get_auth_request con 0x7fd3f80a4a50 auth_method 0 2026-02-25T15:08:00.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.432+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_get_version_reply(handle=1 version=338) ==== 24+0+0 (unknown 1001891302 0 0) 0x7fd418059d70 con 0x7fd428059e10 2026-02-25T15:08:00.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.432+0000 7fd415ffb640 10 monclient: handle_get_version_reply finishing 1 version 338 2026-02-25T15:08:00.441 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.433+0000 7fd415ffb640 10 client.27583.objecter ms_handle_connect 0x7fd3f80a4a50 2026-02-25T15:08:00.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.443+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== osd.11 v1:10.20.193.189:6801/4031575111 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7fd3f80c66b0 con 0x7fd3f80a4a50 2026-02-25T15:08:00.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.443+0000 7fd415ffb640 10 client.27583.objecter ms_dispatch2 0x7fd428000b90 command_reply(tid 1: 0 ) 2026-02-25T15:08:00.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.443+0000 7fd415ffb640 10 client.27583.objecter _finish_command 1 = system:0 2026-02-25T15:08:00.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.443+0000 7fd415ffb640 20 client.27583.objecter put_session s=0x7fd3f80a43a0 osd=11 4 2026-02-25T15:08:00.444 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.443+0000 7fd415ffb640 15 client.27583.objecter _session_command_op_remove 11 1 2026-02-25T15:08:00.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.449+0000 7fd42f832640 10 client.27583.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:08:00.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.449+0000 7fd42f832640 15 client.27583.objecter _session_command_op_assign -1 2 2026-02-25T15:08:00.451 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.449+0000 7fd42f832640 20 client.27583.objecter _get_session s=0x7fd3f80a43a0 osd=11 3 2026-02-25T15:08:00.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.449+0000 7fd42f832640 20 client.27583.objecter put_session s=0x7fd3f80a43a0 osd=11 3 2026-02-25T15:08:00.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.449+0000 7fd42f832640 20 client.27583.objecter _get_session s=0x7fd3f80a43a0 osd=11 3 2026-02-25T15:08:00.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.449+0000 7fd42f832640 15 client.27583.objecter _session_command_op_remove -1 2 2026-02-25T15:08:00.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.449+0000 7fd42f832640 20 client.27583.objecter get_session s=0x7fd3f80a43a0 osd=11 3 2026-02-25T15:08:00.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.449+0000 7fd42f832640 15 client.27583.objecter _session_command_op_assign 11 2 2026-02-25T15:08:00.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.449+0000 7fd42f832640 20 client.27583.objecter put_session s=0x7fd3f80a43a0 osd=11 4 2026-02-25T15:08:00.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.449+0000 7fd42f832640 10 client.27583.objecter _send_command 2 2026-02-25T15:08:00.452 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.449+0000 7fd42f832640 1 -- 10.20.193.90:0/2878767398 --> v1:10.20.193.189:6801/4031575111 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd428192840 con 0x7fd3f80a4a50 2026-02-25T15:08:00.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd415ffb640 1 -- 10.20.193.90:0/2878767398 <== osd.11 v1:10.20.193.189:6801/4031575111 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7fd428192840 con 0x7fd3f80a4a50 2026-02-25T15:08:00.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd415ffb640 10 client.27583.objecter ms_dispatch2 0x7fd428000b90 command_reply(tid 2: 0 ) 2026-02-25T15:08:00.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd415ffb640 10 client.27583.objecter _finish_command 2 = system:0 2026-02-25T15:08:00.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd415ffb640 20 client.27583.objecter put_session s=0x7fd3f80a43a0 osd=11 4 2026-02-25T15:08:00.459 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd415ffb640 15 client.27583.objecter _session_command_op_remove 11 2 2026-02-25T15:08:00.459 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:08:00.460 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-25T15:08:00.460 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:08:00.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 10 librados: watch_flush enter 2026-02-25T15:08:00.461 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 10 librados: watch_flush exit 2026-02-25T15:08:00.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 10 client.27583.objecter close_session for osd.11 2026-02-25T15:08:00.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 1 -- 10.20.193.90:0/2878767398 >> v1:10.20.193.189:6801/4031575111 conn(0x7fd3f80a4a50 legacy=0x7fd3f80c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 20 client.27583.objecter put_session s=0x7fd3f80a43a0 osd=11 1 2026-02-25T15:08:00.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 20 client.27583.objecter shutdown clearing up homeless session... 2026-02-25T15:08:00.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 10 client.27583.objecter successfully canceled tick 2026-02-25T15:08:00.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 1 -- 10.20.193.90:0/2878767398 >> v1:10.20.193.189:6800/3647375982 conn(0x7fd3f807a090 legacy=0x7fd3f809a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 10 monclient: shutdown 2026-02-25T15:08:00.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:00.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 1 -- 10.20.193.90:0/2878767398 >> v1:10.20.193.93:6789/0 conn(0x7fd428059e10 legacy=0x7fd42805d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42dda8640 1 -- 10.20.193.90:0/2878767398 reap_dead start 2026-02-25T15:08:00.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 1 -- 10.20.193.90:0/2878767398 shutdown_connections 2026-02-25T15:08:00.462 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 1 -- 10.20.193.90:0/2878767398 >> 10.20.193.90:0/2878767398 conn(0x7fd4280b2f60 msgr2=0x7fd4281a4900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:00.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 1 -- 10.20.193.90:0/2878767398 shutdown_connections 2026-02-25T15:08:00.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 1 -- 10.20.193.90:0/2878767398 wait complete. 2026-02-25T15:08:00.463 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.457+0000 7fd42f832640 1 librados: shutdown 2026-02-25T15:08:00.467 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2026-02-25T15:08:00.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.534+0000 7f677128a640 10 monclient: get_monmap_and_config 2026-02-25T15:08:00.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.534+0000 7f677128a640 10 monclient: build_initial_monmap 2026-02-25T15:08:00.536 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.534+0000 7f677128a640 10 monclient: monmap: 2026-02-25T15:08:00.537 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:00.537 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.537 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:08:00.535311+0000 2026-02-25T15:08:00.537 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:08:00.535311+0000 2026-02-25T15:08:00.537 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:00.537 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.537 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:00.537 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:00.537 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:00.537 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.543 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.535+0000 7f677128a640 1 Processor -- start 2026-02-25T15:08:00.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.535+0000 7f677128a640 1 -- start start 2026-02-25T15:08:00.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.535+0000 7f677128a640 10 monclient: init 2026-02-25T15:08:00.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.535+0000 7f677128a640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:00.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.535+0000 7f677128a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:08:00.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.535+0000 7f677128a640 10 monclient(hunting): picked mon.noname-c con 0x7f676c157780 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:00.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.535+0000 7f677128a640 10 monclient(hunting): picked mon.noname-b con 0x7f676c157b50 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:00.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.535+0000 7f677128a640 10 monclient(hunting): picked mon.noname-a con 0x7f676c164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:00.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.535+0000 7f677128a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c05de70 con 0x7f676c164d00 2026-02-25T15:08:00.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.535+0000 7f677128a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c055b20 con 0x7f676c157b50 2026-02-25T15:08:00.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.535+0000 7f677128a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c055e00 con 0x7f676c157780 2026-02-25T15:08:00.544 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.535+0000 7f677128a640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:00.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.535+0000 7f677128a640 10 monclient(hunting): authenticate will time out at 3370.135776s 2026-02-25T15:08:00.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.536+0000 7f676a7fc640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f676c157b50 0x7f676c1645f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45946/0 (socket says 10.20.193.90:45946) 2026-02-25T15:08:00.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.536+0000 7f676a7fc640 1 -- 10.20.193.90:0/2544974566 learned_addr learned my addr 10.20.193.90:0/2544974566 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:00.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 1 -- 10.20.193.90:0/2544974566 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3199082741 0 0) 0x7f676c055e00 con 0x7f676c157780 2026-02-25T15:08:00.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 10 monclient(hunting): my global_id is 28724 2026-02-25T15:08:00.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 1 -- 10.20.193.90:0/2544974566 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6758003610 con 0x7f676c157780 2026-02-25T15:08:00.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 1 -- 10.20.193.90:0/2544974566 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3493813255 0 0) 0x7f676c05de70 con 0x7f676c164d00 2026-02-25T15:08:00.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.545 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 10 monclient(hunting): my global_id is 30957 2026-02-25T15:08:00.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 1 -- 10.20.193.90:0/2544974566 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f676c055e00 con 0x7f676c164d00 2026-02-25T15:08:00.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 1 -- 10.20.193.90:0/2544974566 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3325508083 0 0) 0x7f676c055b20 con 0x7f676c157b50 2026-02-25T15:08:00.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 10 monclient(hunting): my global_id is 27586 2026-02-25T15:08:00.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.537+0000 7f6769ffb640 1 -- 10.20.193.90:0/2544974566 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f676c05de70 con 0x7f676c157b50 2026-02-25T15:08:00.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 1 -- 10.20.193.90:0/2544974566 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3485667342 0 0) 0x7f6758003610 con 0x7f676c157780 2026-02-25T15:08:00.546 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 1 -- 10.20.193.90:0/2544974566 >> v1:10.20.193.93:6789/0 conn(0x7f676c157b50 legacy=0x7f676c1645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 1 -- 10.20.193.90:0/2544974566 >> v1:10.20.193.90:6789/0 conn(0x7f676c164d00 legacy=0x7f676c1c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:08:00.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 1 monclient: found mon.noname-c 2026-02-25T15:08:00.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:00.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:08:00.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 1 -- 10.20.193.90:0/2544974566 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676c0560e0 con 0x7f676c157780 2026-02-25T15:08:00.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 10 monclient: _finish_auth 0 2026-02-25T15:08:00.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:08:00.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:00.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 1 -- 10.20.193.90:0/2544974566 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6760002df0 con 0x7f676c157780 2026-02-25T15:08:00.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:00.547 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:08:00.548 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 10 monclient: dump: 2026-02-25T15:08:00.548 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.548 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.548 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.548 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.548 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.548 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.548 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.548 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.548 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.548 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f677128a640 5 monclient: authenticate success, global_id 28724 2026-02-25T15:08:00.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f677128a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:08:00.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 1 -- 10.20.193.90:0/2544974566 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6760002fc0 con 0x7f676c157780 2026-02-25T15:08:00.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:00.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 1 -- 10.20.193.90:0/2544974566 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6760003400 con 0x7f676c157780 2026-02-25T15:08:00.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:00.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:08:00.549 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f6769ffb640 10 monclient: dump: 2026-02-25T15:08:00.549 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.549 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.549 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.549 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.549 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.550 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.550 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.550 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.550 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.550 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f677128a640 10 monclient: get_monmap_and_config success 2026-02-25T15:08:00.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.539+0000 7f677128a640 4 set_mon_vals no callback set 2026-02-25T15:08:00.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.540+0000 7f677128a640 10 monclient: shutdown 2026-02-25T15:08:00.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.540+0000 7f677128a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:00.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.540+0000 7f677128a640 1 -- 10.20.193.90:0/2544974566 >> v1:10.20.193.94:6789/0 conn(0x7f676c157780 legacy=0x7f676c056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.540+0000 7f677128a640 1 -- 10.20.193.90:0/2544974566 shutdown_connections 2026-02-25T15:08:00.550 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.540+0000 7f677128a640 1 -- 10.20.193.90:0/2544974566 >> 10.20.193.90:0/2544974566 conn(0x7f676c0b2ef0 msgr2=0x7f676c0b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:00.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.540+0000 7f677128a640 1 -- 10.20.193.90:0/2544974566 shutdown_connections 2026-02-25T15:08:00.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.540+0000 7f677128a640 1 -- 10.20.193.90:0/2544974566 wait complete. 2026-02-25T15:08:00.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.540+0000 7f677128a640 10 monclient: build_initial_monmap 2026-02-25T15:08:00.551 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.540+0000 7f677128a640 10 monclient: monmap: 2026-02-25T15:08:00.551 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:00.551 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:08:00.551 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:08:00.551 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:08:00.551 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:00.551 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:08:00.551 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:00.552 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:00.552 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:00.552 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.540+0000 7f677128a640 1 librados: starting msgr at 2026-02-25T15:08:00.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.540+0000 7f677128a640 1 librados: starting objecter 2026-02-25T15:08:00.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.540+0000 7f677128a640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:08:00.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.540+0000 7f677128a640 1 Processor -- start 2026-02-25T15:08:00.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f677128a640 1 -- start start 2026-02-25T15:08:00.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f677128a640 1 librados: setting wanted keys 2026-02-25T15:08:00.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f677128a640 1 librados: calling monclient init 2026-02-25T15:08:00.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f677128a640 10 monclient: init 2026-02-25T15:08:00.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f677128a640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:00.552 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f677128a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:08:00.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f677128a640 10 monclient(hunting): picked mon.noname-c con 0x7f676c157780 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:00.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f677128a640 10 monclient(hunting): picked mon.noname-b con 0x7f676c157b50 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:00.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f677128a640 10 monclient(hunting): picked mon.noname-a con 0x7f676c164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:00.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f677128a640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c144460 con 0x7f676c164d00 2026-02-25T15:08:00.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f677128a640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c144740 con 0x7f676c157b50 2026-02-25T15:08:00.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f677128a640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c136820 con 0x7f676c157780 2026-02-25T15:08:00.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f677128a640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:00.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f677128a640 10 monclient(hunting): authenticate will time out at 3370.141418s 2026-02-25T15:08:00.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f676b7fe640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f676c164d00 0x7f676c1375e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:40132/0 (socket says 10.20.193.90:40132) 2026-02-25T15:08:00.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f676b7fe640 1 -- 10.20.193.90:0/879145908 learned_addr learned my addr 10.20.193.90:0/879145908 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:00.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f674b7fe640 10 client.?.objecter ms_handle_connect 0x7f676c164d00 2026-02-25T15:08:00.553 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.541+0000 7f674b7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:00.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.542+0000 7f674b7fe640 10 client.?.objecter ms_handle_connect 0x7f676c157b50 2026-02-25T15:08:00.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.542+0000 7f674b7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:00.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.542+0000 7f674b7fe640 10 client.?.objecter ms_handle_connect 0x7f676c157780 2026-02-25T15:08:00.554 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.542+0000 7f674b7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:00.557 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.542+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 222089663 0 0) 0x7f676c144460 con 0x7f676c164d00 2026-02-25T15:08:00.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.542+0000 7f674b7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.542+0000 7f674b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.542+0000 7f674b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.542+0000 7f674b7fe640 10 monclient(hunting): my global_id is 30960 2026-02-25T15:08:00.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.542+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6738003610 con 0x7f676c164d00 2026-02-25T15:08:00.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.543+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 804552572 0 0) 0x7f676c144740 con 0x7f676c157b50 2026-02-25T15:08:00.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.543+0000 7f674b7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.543+0000 7f674b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.543+0000 7f674b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.543+0000 7f674b7fe640 10 monclient(hunting): my global_id is 27589 2026-02-25T15:08:00.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.543+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f676c144460 con 0x7f676c157b50 2026-02-25T15:08:00.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.543+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 59497204 0 0) 0x7f676c136820 con 0x7f676c157780 2026-02-25T15:08:00.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.543+0000 7f674b7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.543+0000 7f674b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.543+0000 7f674b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.543+0000 7f674b7fe640 10 monclient(hunting): my global_id is 28727 2026-02-25T15:08:00.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.543+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f676c144740 con 0x7f676c157780 2026-02-25T15:08:00.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.545+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 912082842 0 0) 0x7f6738003610 con 0x7f676c164d00 2026-02-25T15:08:00.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.545+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f676c136820 con 0x7f676c164d00 2026-02-25T15:08:00.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.545+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f675c0033f0 con 0x7f676c164d00 2026-02-25T15:08:00.559 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.545+0000 7f674b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:00.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.545+0000 7f674b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:08:00.560 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.545+0000 7f674b7fe640 10 monclient(hunting): dump: 2026-02-25T15:08:00.560 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.560 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.560 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.560 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.560 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.560 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.560 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.560 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.560 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.561 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.545+0000 7f674b7fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:08:00.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.545+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3949831400 0 0) 0x7f676c144740 con 0x7f676c157780 2026-02-25T15:08:00.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.545+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6738003610 con 0x7f676c157780 2026-02-25T15:08:00.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.545+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6760003310 con 0x7f676c157780 2026-02-25T15:08:00.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.545+0000 7f674b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:00.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.545+0000 7f674b7fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:08:00.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.545+0000 7f674b7fe640 10 monclient(hunting): dump: 2026-02-25T15:08:00.561 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.561 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.561 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.562 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.562 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.562 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.562 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.562 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.562 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.562 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.546+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2346813237 0 0) 0x7f676c144460 con 0x7f676c157b50 2026-02-25T15:08:00.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.546+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f676c144740 con 0x7f676c157b50 2026-02-25T15:08:00.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.546+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6754003180 con 0x7f676c157b50 2026-02-25T15:08:00.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.546+0000 7f674b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:00.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.546+0000 7f674b7fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:08:00.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.546+0000 7f674b7fe640 10 monclient(hunting): dump: 2026-02-25T15:08:00.563 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.563 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.563 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.563 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.563 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.563 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.563 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.563 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.563 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.563 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.546+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1377255860 0 0) 0x7f676c136820 con 0x7f676c164d00 2026-02-25T15:08:00.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.546+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 >> v1:10.20.193.94:6789/0 conn(0x7f676c157780 legacy=0x7f676c147850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.546+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 >> v1:10.20.193.93:6789/0 conn(0x7f676c157b50 legacy=0x7f676c135e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.546+0000 7f674b7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:08:00.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.546+0000 7f674b7fe640 1 monclient: found mon.a 2026-02-25T15:08:00.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.546+0000 7f674b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:00.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.546+0000 7f674b7fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:08:00.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.546+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676c136b00 con 0x7f676c164d00 2026-02-25T15:08:00.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f674b7fe640 10 monclient: _finish_auth 0 2026-02-25T15:08:00.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f674b7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:08:00.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f674b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:00.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f677128a640 5 monclient: authenticate success, global_id 30960 2026-02-25T15:08:00.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f677128a640 10 monclient: _renew_subs 2026-02-25T15:08:00.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f677128a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:08:00.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f677128a640 1 -- 10.20.193.90:0/879145908 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f676c136de0 con 0x7f676c164d00 2026-02-25T15:08:00.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f677128a640 10 client.30960.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:08:00.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f677128a640 10 monclient: _renew_subs 2026-02-25T15:08:00.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f677128a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:08:00.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f677128a640 1 -- 10.20.193.90:0/879145908 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f676c0aebe0 con 0x7f676c164d00 2026-02-25T15:08:00.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f677128a640 1 librados: init done 2026-02-25T15:08:00.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f677128a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:00.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f677128a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:00.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f677128a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:08:00.565 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.547+0000 7f677128a640 1 -- 10.20.193.90:0/879145908 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f676c13bfa0 con 0x7f676c164d00 2026-02-25T15:08:00.636 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.635+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f675c003a80 con 0x7f676c164d00 2026-02-25T15:08:00.637 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.635+0000 7f674b7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:00.637 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.635+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f675c003e90 con 0x7f676c164d00 2026-02-25T15:08:00.637 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.635+0000 7f674b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:00.637 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.635+0000 7f674b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:08:00.637 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.635+0000 7f674b7fe640 10 monclient: dump: 2026-02-25T15:08:00.637 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.637 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.637 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.637 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.638 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.638 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.638 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.638 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.638 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.638 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.638 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.635+0000 7f67697fa640 4 set_mon_vals no callback set 2026-02-25T15:08:00.639 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.635+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f675c004070 con 0x7f676c164d00 2026-02-25T15:08:00.639 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.636+0000 7f676affd640 10 monclient: get_auth_request con 0x7f673807a060 auth_method 0 2026-02-25T15:08:00.639 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.636+0000 7f674b7fe640 10 client.30960.objecter ms_handle_connect 0x7f673807a060 2026-02-25T15:08:00.640 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.638+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7f675c095d90 con 0x7f676c164d00 2026-02-25T15:08:00.640 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.638+0000 7f674b7fe640 10 client.30960.objecter ms_dispatch2 0x7f676c000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:08:00.640 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.638+0000 7f674b7fe640 3 client.30960.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:08:00.640 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.638+0000 7f674b7fe640 3 client.30960.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:08:00.640 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.638+0000 7f674b7fe640 20 client.30960.objecter dump_active .. 0 homeless 2026-02-25T15:08:00.640 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.638+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f676c0aebe0 con 0x7f676c164d00 2026-02-25T15:08:00.641 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.638+0000 7f674b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:00.641 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.638+0000 7f674b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:08:00.748 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.746+0000 7f677128a640 1 -- 10.20.193.90:0/879145908 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "11", "target": ["mon-mgr", ""]}) -- 0x7f676c13bcc0 con 0x7f673807a060 2026-02-25T15:08:00.748 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.746+0000 7f674b7fe640 1 -- 10.20.193.90:0/879145908 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 11 to deep-scrub) ==== 42+0+0 (unknown 2412662492 0 0) 0x7f676c13bcc0 con 0x7f673807a060 2026-02-25T15:08:00.748 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 11 to deep-scrub 2026-02-25T15:08:00.749 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.748+0000 7f677128a640 10 librados: watch_flush enter 2026-02-25T15:08:00.749 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.748+0000 7f677128a640 10 librados: watch_flush exit 2026-02-25T15:08:00.749 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.748+0000 7f677128a640 20 client.30960.objecter shutdown clearing up homeless session... 2026-02-25T15:08:00.749 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.748+0000 7f677128a640 10 client.30960.objecter successfully canceled tick 2026-02-25T15:08:00.749 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.748+0000 7f677128a640 1 -- 10.20.193.90:0/879145908 >> v1:10.20.193.189:6800/3647375982 conn(0x7f673807a060 legacy=0x7f673809a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.750 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.748+0000 7f677128a640 10 monclient: shutdown 2026-02-25T15:08:00.750 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.748+0000 7f677128a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:00.750 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.748+0000 7f677128a640 1 -- 10.20.193.90:0/879145908 >> v1:10.20.193.90:6789/0 conn(0x7f676c164d00 legacy=0x7f676c1375e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.750 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.748+0000 7f677128a640 1 -- 10.20.193.90:0/879145908 shutdown_connections 2026-02-25T15:08:00.750 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.748+0000 7f677128a640 1 -- 10.20.193.90:0/879145908 >> 10.20.193.90:0/879145908 conn(0x7f676c0b2ef0 msgr2=0x7f676c0b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:00.750 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.748+0000 7f677128a640 1 -- 10.20.193.90:0/879145908 shutdown_connections 2026-02-25T15:08:00.750 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.748+0000 7f677128a640 1 -- 10.20.193.90:0/879145908 wait complete. 2026-02-25T15:08:00.750 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.748+0000 7f677128a640 1 librados: shutdown 2026-02-25T15:08:00.758 INFO:tasks.ceph:Scrubbing osd.15 2026-02-25T15:08:00.758 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 config set osd_debug_deep_scrub_sleep 0 2026-02-25T15:08:00.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.826+0000 7ff43878b640 10 monclient: get_monmap_and_config 2026-02-25T15:08:00.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.826+0000 7ff43878b640 10 monclient: build_initial_monmap 2026-02-25T15:08:00.828 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.826+0000 7ff43878b640 10 monclient: monmap: 2026-02-25T15:08:00.828 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:00.828 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.828 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:08:00.826996+0000 2026-02-25T15:08:00.828 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:08:00.826996+0000 2026-02-25T15:08:00.828 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:00.829 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.829 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:00.829 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:00.829 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:00.829 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.826+0000 7ff43878b640 1 Processor -- start 2026-02-25T15:08:00.831 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.826+0000 7ff43878b640 1 -- start start 2026-02-25T15:08:00.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.826+0000 7ff43878b640 10 monclient: init 2026-02-25T15:08:00.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.827+0000 7ff43878b640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:00.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.827+0000 7ff43878b640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:08:00.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.827+0000 7ff43878b640 10 monclient(hunting): picked mon.noname-b con 0x7ff43015e2a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:00.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.827+0000 7ff43878b640 10 monclient(hunting): picked mon.noname-c con 0x7ff430059e10 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:00.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.827+0000 7ff43878b640 10 monclient(hunting): picked mon.noname-a con 0x7ff4301a5660 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:00.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.827+0000 7ff43878b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff430162df0 con 0x7ff4301a5660 2026-02-25T15:08:00.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.827+0000 7ff43878b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4301630d0 con 0x7ff43015e2a0 2026-02-25T15:08:00.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.827+0000 7ff43878b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4301633b0 con 0x7ff430059e10 2026-02-25T15:08:00.832 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.827+0000 7ff43878b640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:00.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.827+0000 7ff43878b640 10 monclient(hunting): authenticate will time out at 3370.427439s 2026-02-25T15:08:00.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.827+0000 7ff436500640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7ff43015e2a0 0x7ff43005da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45956/0 (socket says 10.20.193.90:45956) 2026-02-25T15:08:00.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.827+0000 7ff436500640 1 -- 10.20.193.90:0/243305611 learned_addr learned my addr 10.20.193.90:0/243305611 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:00.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 1 -- 10.20.193.90:0/243305611 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2483681491 0 0) 0x7ff4301630d0 con 0x7ff43015e2a0 2026-02-25T15:08:00.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 10 monclient(hunting): my global_id is 27592 2026-02-25T15:08:00.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 1 -- 10.20.193.90:0/243305611 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff418003610 con 0x7ff43015e2a0 2026-02-25T15:08:00.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 1 -- 10.20.193.90:0/243305611 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3881593825 0 0) 0x7ff4301633b0 con 0x7ff430059e10 2026-02-25T15:08:00.833 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 10 monclient(hunting): my global_id is 28730 2026-02-25T15:08:00.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 1 -- 10.20.193.90:0/243305611 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4301630d0 con 0x7ff430059e10 2026-02-25T15:08:00.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 1 -- 10.20.193.90:0/243305611 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1625819837 0 0) 0x7ff430162df0 con 0x7ff4301a5660 2026-02-25T15:08:00.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 10 monclient(hunting): my global_id is 30963 2026-02-25T15:08:00.834 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.829+0000 7ff4354fe640 1 -- 10.20.193.90:0/243305611 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4301633b0 con 0x7ff4301a5660 2026-02-25T15:08:00.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff4354fe640 1 -- 10.20.193.90:0/243305611 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 719260484 0 0) 0x7ff418003610 con 0x7ff43015e2a0 2026-02-25T15:08:00.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff4354fe640 1 -- 10.20.193.90:0/243305611 >> v1:10.20.193.94:6789/0 conn(0x7ff430059e10 legacy=0x7ff43015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff4354fe640 1 -- 10.20.193.90:0/243305611 >> v1:10.20.193.90:6789/0 conn(0x7ff4301a5660 legacy=0x7ff4301c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff4354fe640 10 monclient: _finish_hunting 0 2026-02-25T15:08:00.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff4354fe640 1 monclient: found mon.noname-b 2026-02-25T15:08:00.840 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff4354fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:00.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff4354fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff4354fe640 1 -- 10.20.193.90:0/243305611 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff430163690 con 0x7ff43015e2a0 2026-02-25T15:08:00.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff4354fe640 10 monclient: _finish_auth 0 2026-02-25T15:08:00.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff4354fe640 10 monclient: _check_auth_tickets 2026-02-25T15:08:00.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff4354fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:00.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff4354fe640 1 -- 10.20.193.90:0/243305611 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff420002df0 con 0x7ff43015e2a0 2026-02-25T15:08:00.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff4354fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:00.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff4354fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:08:00.841 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff4354fe640 10 monclient: dump: 2026-02-25T15:08:00.841 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.841 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.841 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.842 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.842 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.842 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.842 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.842 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.842 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.842 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff43878b640 5 monclient: authenticate success, global_id 27592 2026-02-25T15:08:00.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.831+0000 7ff43878b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:08:00.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff4354fe640 1 -- 10.20.193.90:0/243305611 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff420002fc0 con 0x7ff43015e2a0 2026-02-25T15:08:00.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff4354fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:00.842 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff4354fe640 1 -- 10.20.193.90:0/243305611 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff420003400 con 0x7ff43015e2a0 2026-02-25T15:08:00.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff4354fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:00.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff4354fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:08:00.843 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff4354fe640 10 monclient: dump: 2026-02-25T15:08:00.843 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.843 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.843 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.843 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.843 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.843 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.843 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.843 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.843 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.843 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff43878b640 10 monclient: get_monmap_and_config success 2026-02-25T15:08:00.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff43878b640 4 set_mon_vals no callback set 2026-02-25T15:08:00.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff43878b640 10 monclient: shutdown 2026-02-25T15:08:00.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff43878b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:00.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff43878b640 1 -- 10.20.193.90:0/243305611 >> v1:10.20.193.93:6789/0 conn(0x7ff43015e2a0 legacy=0x7ff43005da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff43878b640 1 -- 10.20.193.90:0/243305611 shutdown_connections 2026-02-25T15:08:00.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff43878b640 1 -- 10.20.193.90:0/243305611 >> 10.20.193.90:0/243305611 conn(0x7ff4300b2f60 msgr2=0x7ff4300b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:00.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff43878b640 1 -- 10.20.193.90:0/243305611 shutdown_connections 2026-02-25T15:08:00.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff43878b640 1 -- 10.20.193.90:0/243305611 wait complete. 2026-02-25T15:08:00.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff43878b640 10 monclient: build_initial_monmap 2026-02-25T15:08:00.844 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.832+0000 7ff43878b640 10 monclient: monmap: 2026-02-25T15:08:00.845 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:00.845 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:08:00.845 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:08:00.845 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:08:00.845 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:00.845 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:08:00.845 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:00.845 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:00.845 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:00.845 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 1 librados: starting msgr at 2026-02-25T15:08:00.845 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 1 librados: starting objecter 2026-02-25T15:08:00.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:08:00.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 1 Processor -- start 2026-02-25T15:08:00.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 1 -- start start 2026-02-25T15:08:00.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 1 librados: setting wanted keys 2026-02-25T15:08:00.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 1 librados: calling monclient init 2026-02-25T15:08:00.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 10 monclient: init 2026-02-25T15:08:00.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:00.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:08:00.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 10 monclient(hunting): picked mon.noname-a con 0x7ff430059e10 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:00.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 10 monclient(hunting): picked mon.noname-b con 0x7ff43015e2a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:00.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 10 monclient(hunting): picked mon.noname-c con 0x7ff4301a5660 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:00.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4301916c0 con 0x7ff430059e10 2026-02-25T15:08:00.846 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4301919a0 con 0x7ff43015e2a0 2026-02-25T15:08:00.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff43005b9f0 con 0x7ff4301a5660 2026-02-25T15:08:00.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:00.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff43878b640 10 monclient(hunting): authenticate will time out at 3370.433530s 2026-02-25T15:08:00.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff435cff640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7ff43015e2a0 0x7ff43005d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45968/0 (socket says 10.20.193.90:45968) 2026-02-25T15:08:00.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff435cff640 1 -- 10.20.193.90:0/1698802418 learned_addr learned my addr 10.20.193.90:0/1698802418 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:00.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff416ffd640 10 client.?.objecter ms_handle_connect 0x7ff43015e2a0 2026-02-25T15:08:00.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.833+0000 7ff416ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:00.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.834+0000 7ff416ffd640 10 client.?.objecter ms_handle_connect 0x7ff430059e10 2026-02-25T15:08:00.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.834+0000 7ff416ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:00.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.834+0000 7ff416ffd640 10 client.?.objecter ms_handle_connect 0x7ff4301a5660 2026-02-25T15:08:00.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.834+0000 7ff416ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:00.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.834+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3562638109 0 0) 0x7ff4301919a0 con 0x7ff43015e2a0 2026-02-25T15:08:00.847 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.834+0000 7ff416ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.834+0000 7ff416ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.834+0000 7ff416ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.834+0000 7ff416ffd640 10 monclient(hunting): my global_id is 27595 2026-02-25T15:08:00.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.834+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff400003610 con 0x7ff43015e2a0 2026-02-25T15:08:00.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.835+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 223605451 0 0) 0x7ff4301916c0 con 0x7ff430059e10 2026-02-25T15:08:00.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.835+0000 7ff416ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.835+0000 7ff416ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.835+0000 7ff416ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.835+0000 7ff416ffd640 10 monclient(hunting): my global_id is 30966 2026-02-25T15:08:00.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.835+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4301919a0 con 0x7ff430059e10 2026-02-25T15:08:00.848 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.835+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3289707477 0 0) 0x7ff43005b9f0 con 0x7ff4301a5660 2026-02-25T15:08:00.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.835+0000 7ff416ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.835+0000 7ff416ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.835+0000 7ff416ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.835+0000 7ff416ffd640 10 monclient(hunting): my global_id is 28733 2026-02-25T15:08:00.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.835+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4301916c0 con 0x7ff4301a5660 2026-02-25T15:08:00.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.837+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 347205412 0 0) 0x7ff400003610 con 0x7ff43015e2a0 2026-02-25T15:08:00.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.837+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff43005b9f0 con 0x7ff43015e2a0 2026-02-25T15:08:00.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.837+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff4240030f0 con 0x7ff43015e2a0 2026-02-25T15:08:00.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.837+0000 7ff416ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:00.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.837+0000 7ff416ffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:08:00.849 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.837+0000 7ff416ffd640 10 monclient(hunting): dump: 2026-02-25T15:08:00.849 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.850 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.850 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.850 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.850 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.850 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.850 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.850 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.850 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.850 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.850 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.837+0000 7ff416ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:08:00.850 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.837+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3238306178 0 0) 0x7ff4301916c0 con 0x7ff4301a5660 2026-02-25T15:08:00.850 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.837+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff400003610 con 0x7ff4301a5660 2026-02-25T15:08:00.851 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.837+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff42c003410 con 0x7ff4301a5660 2026-02-25T15:08:00.851 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.837+0000 7ff416ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:00.851 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.837+0000 7ff416ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:08:00.851 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.837+0000 7ff416ffd640 10 monclient(hunting): dump: 2026-02-25T15:08:00.851 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.851 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.851 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.851 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.851 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.851 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.851 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.852 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.852 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.852 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.838+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 238140423 0 0) 0x7ff4301919a0 con 0x7ff430059e10 2026-02-25T15:08:00.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.838+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff4301916c0 con 0x7ff430059e10 2026-02-25T15:08:00.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.838+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff420003310 con 0x7ff430059e10 2026-02-25T15:08:00.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.838+0000 7ff416ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:00.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.838+0000 7ff416ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:08:00.852 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.838+0000 7ff416ffd640 10 monclient(hunting): dump: 2026-02-25T15:08:00.852 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.852 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.852 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.853 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.853 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.853 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.853 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.853 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.853 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.853 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1355245969 0 0) 0x7ff43005b9f0 con 0x7ff43015e2a0 2026-02-25T15:08:00.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 >> v1:10.20.193.94:6789/0 conn(0x7ff4301a5660 legacy=0x7ff430190bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 >> v1:10.20.193.90:6789/0 conn(0x7ff430059e10 legacy=0x7ff43005c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff416ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:08:00.856 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff416ffd640 1 monclient: found mon.b 2026-02-25T15:08:00.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff416ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:00.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff416ffd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43005bcd0 con 0x7ff43015e2a0 2026-02-25T15:08:00.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff416ffd640 10 monclient: _finish_auth 0 2026-02-25T15:08:00.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff416ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:08:00.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff416ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:00.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff43878b640 5 monclient: authenticate success, global_id 27595 2026-02-25T15:08:00.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff43878b640 10 monclient: _renew_subs 2026-02-25T15:08:00.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff43878b640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff43878b640 1 -- 10.20.193.90:0/1698802418 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff43005bfb0 con 0x7ff43015e2a0 2026-02-25T15:08:00.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff43878b640 10 client.27595.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:08:00.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff43878b640 10 monclient: _renew_subs 2026-02-25T15:08:00.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff43878b640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff43878b640 1 -- 10.20.193.90:0/1698802418 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff4300a7870 con 0x7ff43015e2a0 2026-02-25T15:08:00.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff43878b640 1 librados: init done 2026-02-25T15:08:00.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff43878b640 10 client.27595.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:00.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff43878b640 15 client.27595.objecter _session_command_op_assign -1 1 2026-02-25T15:08:00.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff43878b640 20 client.27595.objecter _get_session osd=-1 returning homeless 2026-02-25T15:08:00.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff43878b640 10 client.27595.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:08:00.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff43878b640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.839+0000 7ff43878b640 1 -- 10.20.193.90:0/1698802418 --> v1:10.20.193.93:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7ff43005cd70 con 0x7ff43015e2a0 2026-02-25T15:08:00.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.847+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff43005cd70 con 0x7ff43015e2a0 2026-02-25T15:08:00.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.847+0000 7ff416ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:00.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.847+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7ff4240039b0 con 0x7ff43015e2a0 2026-02-25T15:08:00.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.847+0000 7ff416ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:00.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.847+0000 7ff416ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:08:00.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.847+0000 7ff434cfd640 4 set_mon_vals no callback set 2026-02-25T15:08:00.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.847+0000 7ff416ffd640 10 monclient: dump: 2026-02-25T15:08:00.859 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.859 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.859 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.859 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.859 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.859 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.859 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.859 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.859 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.859 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.848+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7ff424003bc0 con 0x7ff43015e2a0 2026-02-25T15:08:00.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.849+0000 7ff436500640 10 monclient: get_auth_request con 0x7ff40007a040 auth_method 0 2026-02-25T15:08:00.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.849+0000 7ff416ffd640 10 client.27595.objecter ms_handle_connect 0x7ff40007a040 2026-02-25T15:08:00.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7ff4240039b0 con 0x7ff43015e2a0 2026-02-25T15:08:00.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 10 client.27595.objecter ms_dispatch2 0x7ff430000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:08:00.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 3 client.27595.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:08:00.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 3 client.27595.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:08:00.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 10 client.27595.objecter checking command 1 2026-02-25T15:08:00.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 20 client.27595.objecter _get_session s=0x7ff4000a4350 osd=15 3 2026-02-25T15:08:00.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 20 client.27595.objecter put_session s=0x7ff4000a4350 osd=15 3 2026-02-25T15:08:00.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 15 client.27595.objecter _session_command_op_remove -1 1 2026-02-25T15:08:00.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 20 client.27595.objecter _get_session s=0x7ff4000a4350 osd=15 3 2026-02-25T15:08:00.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 20 client.27595.objecter get_session s=0x7ff4000a4350 osd=15 3 2026-02-25T15:08:00.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 15 client.27595.objecter _session_command_op_assign 15 1 2026-02-25T15:08:00.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 20 client.27595.objecter put_session s=0x7ff4000a4350 osd=15 4 2026-02-25T15:08:00.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 10 client.27595.objecter _send_command 1 2026-02-25T15:08:00.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 --> v1:10.20.193.189:6805/4039858964 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff4000c6660 con 0x7ff4000a4a00 2026-02-25T15:08:00.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 20 client.27595.objecter dump_active .. 0 homeless 2026-02-25T15:08:00.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_get_version_reply(handle=1 version=338) ==== 24+0+0 (unknown 1001891302 0 0) 0x7ff42405aea0 con 0x7ff43015e2a0 2026-02-25T15:08:00.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.850+0000 7ff416ffd640 10 monclient: handle_get_version_reply finishing 1 version 338 2026-02-25T15:08:00.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.851+0000 7ff436d01640 10 monclient: get_auth_request con 0x7ff4000a4a00 auth_method 0 2026-02-25T15:08:00.861 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.852+0000 7ff416ffd640 10 client.27595.objecter ms_handle_connect 0x7ff4000a4a00 2026-02-25T15:08:00.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.861+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== osd.15 v1:10.20.193.189:6805/4039858964 1 ==== command_reply(tid 1: 0 ) ==== 8+0+32513 (unknown 0 0 875166375) 0x7ff4000c6660 con 0x7ff4000a4a00 2026-02-25T15:08:00.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.861+0000 7ff416ffd640 10 client.27595.objecter ms_dispatch2 0x7ff430000b90 command_reply(tid 1: 0 ) 2026-02-25T15:08:00.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.861+0000 7ff416ffd640 10 client.27595.objecter _finish_command 1 = system:0 2026-02-25T15:08:00.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.862+0000 7ff416ffd640 20 client.27595.objecter put_session s=0x7ff4000a4350 osd=15 4 2026-02-25T15:08:00.863 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.862+0000 7ff416ffd640 15 client.27595.objecter _session_command_op_remove 15 1 2026-02-25T15:08:00.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.869+0000 7ff43878b640 10 client.27595.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-25T15:08:00.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.869+0000 7ff43878b640 15 client.27595.objecter _session_command_op_assign -1 2 2026-02-25T15:08:00.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.869+0000 7ff43878b640 20 client.27595.objecter _get_session s=0x7ff4000a4350 osd=15 3 2026-02-25T15:08:00.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.869+0000 7ff43878b640 20 client.27595.objecter put_session s=0x7ff4000a4350 osd=15 3 2026-02-25T15:08:00.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.869+0000 7ff43878b640 20 client.27595.objecter _get_session s=0x7ff4000a4350 osd=15 3 2026-02-25T15:08:00.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.869+0000 7ff43878b640 15 client.27595.objecter _session_command_op_remove -1 2 2026-02-25T15:08:00.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.869+0000 7ff43878b640 20 client.27595.objecter get_session s=0x7ff4000a4350 osd=15 3 2026-02-25T15:08:00.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.869+0000 7ff43878b640 15 client.27595.objecter _session_command_op_assign 15 2 2026-02-25T15:08:00.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.869+0000 7ff43878b640 20 client.27595.objecter put_session s=0x7ff4000a4350 osd=15 4 2026-02-25T15:08:00.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.869+0000 7ff43878b640 10 client.27595.objecter _send_command 2 2026-02-25T15:08:00.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.869+0000 7ff43878b640 1 -- 10.20.193.90:0/1698802418 --> v1:10.20.193.189:6805/4039858964 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff4301922f0 con 0x7ff4000a4a00 2026-02-25T15:08:00.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff416ffd640 1 -- 10.20.193.90:0/1698802418 <== osd.15 v1:10.20.193.189:6805/4039858964 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7ff4301922f0 con 0x7ff4000a4a00 2026-02-25T15:08:00.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff416ffd640 10 client.27595.objecter ms_dispatch2 0x7ff430000b90 command_reply(tid 2: 0 ) 2026-02-25T15:08:00.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff416ffd640 10 client.27595.objecter _finish_command 2 = system:0 2026-02-25T15:08:00.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff416ffd640 20 client.27595.objecter put_session s=0x7ff4000a4350 osd=15 4 2026-02-25T15:08:00.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff416ffd640 15 client.27595.objecter _session_command_op_remove 15 2 2026-02-25T15:08:00.880 INFO:teuthology.orchestra.run.trial090.stdout:{ 2026-02-25T15:08:00.880 INFO:teuthology.orchestra.run.trial090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-25T15:08:00.880 INFO:teuthology.orchestra.run.trial090.stdout:} 2026-02-25T15:08:00.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff43878b640 10 librados: watch_flush enter 2026-02-25T15:08:00.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff43878b640 10 librados: watch_flush exit 2026-02-25T15:08:00.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff43878b640 10 client.27595.objecter close_session for osd.15 2026-02-25T15:08:00.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff43878b640 1 -- 10.20.193.90:0/1698802418 >> v1:10.20.193.189:6805/4039858964 conn(0x7ff4000a4a00 legacy=0x7ff4000c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff43878b640 20 client.27595.objecter put_session s=0x7ff4000a4350 osd=15 1 2026-02-25T15:08:00.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff43878b640 20 client.27595.objecter shutdown clearing up homeless session... 2026-02-25T15:08:00.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff43878b640 10 client.27595.objecter successfully canceled tick 2026-02-25T15:08:00.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff43878b640 1 -- 10.20.193.90:0/1698802418 >> v1:10.20.193.189:6800/3647375982 conn(0x7ff40007a040 legacy=0x7ff40009a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff43878b640 10 monclient: shutdown 2026-02-25T15:08:00.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff43878b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:00.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff43878b640 1 -- 10.20.193.90:0/1698802418 >> v1:10.20.193.93:6789/0 conn(0x7ff43015e2a0 legacy=0x7ff43005d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff436d01640 1 -- 10.20.193.90:0/1698802418 reap_dead start 2026-02-25T15:08:00.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff43878b640 1 -- 10.20.193.90:0/1698802418 shutdown_connections 2026-02-25T15:08:00.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff43878b640 1 -- 10.20.193.90:0/1698802418 >> 10.20.193.90:0/1698802418 conn(0x7ff4300b2f60 msgr2=0x7ff430192810 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:00.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.877+0000 7ff43878b640 1 -- 10.20.193.90:0/1698802418 shutdown_connections 2026-02-25T15:08:00.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.878+0000 7ff43878b640 1 -- 10.20.193.90:0/1698802418 wait complete. 2026-02-25T15:08:00.883 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.878+0000 7ff43878b640 1 librados: shutdown 2026-02-25T15:08:00.887 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 15 2026-02-25T15:08:00.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.955+0000 7fabcce04640 10 monclient: get_monmap_and_config 2026-02-25T15:08:00.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.955+0000 7fabcce04640 10 monclient: build_initial_monmap 2026-02-25T15:08:00.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.955+0000 7fabcce04640 10 monclient: monmap: 2026-02-25T15:08:00.957 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:00.957 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.957 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:08:00.955897+0000 2026-02-25T15:08:00.957 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:08:00.955897+0000 2026-02-25T15:08:00.957 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:00.957 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.957 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:00.957 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:00.958 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:00.958 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.955+0000 7fabcce04640 1 Processor -- start 2026-02-25T15:08:00.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.955+0000 7fabcce04640 1 -- start start 2026-02-25T15:08:00.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.955+0000 7fabcce04640 10 monclient: init 2026-02-25T15:08:00.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.956+0000 7fabcce04640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:00.960 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.956+0000 7fabcce04640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-25T15:08:00.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.956+0000 7fabcce04640 10 monclient(hunting): picked mon.noname-b con 0x7fabc8157780 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:00.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.956+0000 7fabcce04640 10 monclient(hunting): picked mon.noname-c con 0x7fabc8157b50 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:00.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.956+0000 7fabcce04640 10 monclient(hunting): picked mon.noname-a con 0x7fabc8164d00 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:00.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.956+0000 7fabcce04640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc805de70 con 0x7fabc8164d00 2026-02-25T15:08:00.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.956+0000 7fabcce04640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc8055b20 con 0x7fabc8157780 2026-02-25T15:08:00.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.956+0000 7fabcce04640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc8055e00 con 0x7fabc8157b50 2026-02-25T15:08:00.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.956+0000 7fabcce04640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:00.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.956+0000 7fabcce04640 10 monclient(hunting): authenticate will time out at 3370.556268s 2026-02-25T15:08:00.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.956+0000 7fabc6575640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7fabc8157780 0x7fabc8056ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:45982/0 (socket says 10.20.193.90:45982) 2026-02-25T15:08:00.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.956+0000 7fabc6575640 1 -- 10.20.193.90:0/6420430 learned_addr learned my addr 10.20.193.90:0/6420430 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:00.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 1 -- 10.20.193.90:0/6420430 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2145611174 0 0) 0x7fabc8055e00 con 0x7fabc8157b50 2026-02-25T15:08:00.961 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 10 monclient(hunting): my global_id is 28736 2026-02-25T15:08:00.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 1 -- 10.20.193.90:0/6420430 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabb4003610 con 0x7fabc8157b50 2026-02-25T15:08:00.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 1 -- 10.20.193.90:0/6420430 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1826823682 0 0) 0x7fabc8055b20 con 0x7fabc8157780 2026-02-25T15:08:00.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 10 monclient(hunting): my global_id is 27598 2026-02-25T15:08:00.962 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 1 -- 10.20.193.90:0/6420430 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabc8055e00 con 0x7fabc8157780 2026-02-25T15:08:00.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 1 -- 10.20.193.90:0/6420430 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2238651708 0 0) 0x7fabc805de70 con 0x7fabc8164d00 2026-02-25T15:08:00.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 10 monclient(hunting): my global_id is 30969 2026-02-25T15:08:00.963 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.958+0000 7fabc5573640 1 -- 10.20.193.90:0/6420430 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabc8055b20 con 0x7fabc8164d00 2026-02-25T15:08:00.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabc5573640 1 -- 10.20.193.90:0/6420430 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1581903088 0 0) 0x7fabc8055e00 con 0x7fabc8157780 2026-02-25T15:08:00.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabc5573640 1 -- 10.20.193.90:0/6420430 >> v1:10.20.193.94:6789/0 conn(0x7fabc8157b50 legacy=0x7fabc81645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.970 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabc5573640 1 -- 10.20.193.90:0/6420430 >> v1:10.20.193.90:6789/0 conn(0x7fabc8164d00 legacy=0x7fabc81c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabc5573640 10 monclient: _finish_hunting 0 2026-02-25T15:08:00.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabc5573640 1 monclient: found mon.noname-b 2026-02-25T15:08:00.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabc5573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:00.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabc5573640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabc5573640 1 -- 10.20.193.90:0/6420430 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc80560e0 con 0x7fabc8157780 2026-02-25T15:08:00.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabc5573640 10 monclient: _finish_auth 0 2026-02-25T15:08:00.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabc5573640 10 monclient: _check_auth_tickets 2026-02-25T15:08:00.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabc5573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:00.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabcce04640 5 monclient: authenticate success, global_id 27598 2026-02-25T15:08:00.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabcce04640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:08:00.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabc5573640 1 -- 10.20.193.90:0/6420430 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fabbc002d70 con 0x7fabc8157780 2026-02-25T15:08:00.971 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabc5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:00.972 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabc5573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:08:00.972 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabc5573640 10 monclient: dump: 2026-02-25T15:08:00.972 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.972 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.972 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.972 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.972 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.972 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.972 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.972 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.972 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.972 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.959+0000 7fabcce04640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:08:00.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.960+0000 7fabc5573640 1 -- 10.20.193.90:0/6420430 <== mon.1 v1:10.20.193.93:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fabbc002fc0 con 0x7fabc8157780 2026-02-25T15:08:00.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.960+0000 7fabc5573640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:00.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.960+0000 7fabc5573640 1 -- 10.20.193.90:0/6420430 <== mon.1 v1:10.20.193.93:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fabbc0033c0 con 0x7fabc8157780 2026-02-25T15:08:00.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.960+0000 7fabc5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:00.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.960+0000 7fabc5573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:08:00.973 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.960+0000 7fabc5573640 10 monclient: dump: 2026-02-25T15:08:00.973 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.973 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.973 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.973 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.973 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.973 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.974 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.974 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.974 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.974 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.960+0000 7fabcce04640 10 monclient: get_monmap_and_config success 2026-02-25T15:08:00.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.960+0000 7fabcce04640 4 set_mon_vals no callback set 2026-02-25T15:08:00.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.960+0000 7fabcce04640 10 monclient: shutdown 2026-02-25T15:08:00.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.960+0000 7fabcce04640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:00.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.960+0000 7fabcce04640 1 -- 10.20.193.90:0/6420430 >> v1:10.20.193.93:6789/0 conn(0x7fabc8157780 legacy=0x7fabc8056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.960+0000 7fabcce04640 1 -- 10.20.193.90:0/6420430 shutdown_connections 2026-02-25T15:08:00.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.960+0000 7fabcce04640 1 -- 10.20.193.90:0/6420430 >> 10.20.193.90:0/6420430 conn(0x7fabc80b2ef0 msgr2=0x7fabc80b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:00.974 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.960+0000 7fabcce04640 1 -- 10.20.193.90:0/6420430 shutdown_connections 2026-02-25T15:08:00.975 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.961+0000 7fabcce04640 1 -- 10.20.193.90:0/6420430 wait complete. 2026-02-25T15:08:00.975 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.961+0000 7fabcce04640 10 monclient: build_initial_monmap 2026-02-25T15:08:00.975 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.961+0000 7fabcce04640 10 monclient: monmap: 2026-02-25T15:08:00.975 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:00.975 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:08:00.975 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:08:00.975 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:08:00.975 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:00.975 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:08:00.975 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:00.975 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:00.975 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:00.976 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.976 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.961+0000 7fabcce04640 1 librados: starting msgr at 2026-02-25T15:08:00.976 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.961+0000 7fabcce04640 1 librados: starting objecter 2026-02-25T15:08:00.976 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.961+0000 7fabcce04640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:08:00.976 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.961+0000 7fabcce04640 1 Processor -- start 2026-02-25T15:08:00.976 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.961+0000 7fabcce04640 1 -- start start 2026-02-25T15:08:00.976 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.961+0000 7fabcce04640 1 librados: setting wanted keys 2026-02-25T15:08:00.976 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.961+0000 7fabcce04640 1 librados: calling monclient init 2026-02-25T15:08:00.976 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.961+0000 7fabcce04640 10 monclient: init 2026-02-25T15:08:00.976 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabcce04640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:00.976 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabcce04640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:08:00.976 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabcce04640 10 monclient(hunting): picked mon.noname-a con 0x7fabc8157780 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:00.976 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabcce04640 10 monclient(hunting): picked mon.noname-b con 0x7fabc8157b50 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:00.977 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabcce04640 10 monclient(hunting): picked mon.noname-c con 0x7fabc8164d00 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:00.977 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabcce04640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc81372e0 con 0x7fabc8157780 2026-02-25T15:08:00.977 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabcce04640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc80a87c0 con 0x7fabc8157b50 2026-02-25T15:08:00.977 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabcce04640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc80a8aa0 con 0x7fabc8164d00 2026-02-25T15:08:00.977 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabcce04640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:00.977 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabcce04640 10 monclient(hunting): authenticate will time out at 3370.562284s 2026-02-25T15:08:00.977 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabc6575640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7fabc8157780 0x7fabc80a7e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:40156/0 (socket says 10.20.193.90:40156) 2026-02-25T15:08:00.977 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabc6575640 1 -- 10.20.193.90:0/1218847322 learned_addr learned my addr 10.20.193.90:0/1218847322 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:00.977 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabaeffd640 10 client.?.objecter ms_handle_connect 0x7fabc8157780 2026-02-25T15:08:00.977 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabaeffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:00.977 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabaeffd640 10 client.?.objecter ms_handle_connect 0x7fabc8157b50 2026-02-25T15:08:00.977 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.962+0000 7fabaeffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:00.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.963+0000 7fabaeffd640 10 client.?.objecter ms_handle_connect 0x7fabc8164d00 2026-02-25T15:08:00.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.963+0000 7fabaeffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:00.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.963+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4120893982 0 0) 0x7fabc81372e0 con 0x7fabc8157780 2026-02-25T15:08:00.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.963+0000 7fabaeffd640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.963+0000 7fabaeffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.963+0000 7fabaeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.963+0000 7fabaeffd640 10 monclient(hunting): my global_id is 30972 2026-02-25T15:08:00.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.963+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab94003610 con 0x7fabc8157780 2026-02-25T15:08:00.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.963+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2780692054 0 0) 0x7fabc80a87c0 con 0x7fabc8157b50 2026-02-25T15:08:00.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.963+0000 7fabaeffd640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.963+0000 7fabaeffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.963+0000 7fabaeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.978 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.963+0000 7fabaeffd640 10 monclient(hunting): my global_id is 27601 2026-02-25T15:08:00.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.963+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabc81372e0 con 0x7fabc8157b50 2026-02-25T15:08:00.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.964+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4149783837 0 0) 0x7fabc80a8aa0 con 0x7fabc8164d00 2026-02-25T15:08:00.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.964+0000 7fabaeffd640 10 monclient(hunting): _negotiate 2026-02-25T15:08:00.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.964+0000 7fabaeffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:00.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.964+0000 7fabaeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:00.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.964+0000 7fabaeffd640 10 monclient(hunting): my global_id is 28739 2026-02-25T15:08:00.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.964+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabc80a87c0 con 0x7fabc8164d00 2026-02-25T15:08:00.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3478732193 0 0) 0x7fabc81372e0 con 0x7fabc8157b50 2026-02-25T15:08:00.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fabc80a8aa0 con 0x7fabc8157b50 2026-02-25T15:08:00.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fabb00030b0 con 0x7fabc8157b50 2026-02-25T15:08:00.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:00.979 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:08:00.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 10 monclient(hunting): dump: 2026-02-25T15:08:00.980 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.980 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.980 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.980 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.980 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.980 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.980 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.980 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.980 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.980 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.980 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:08:00.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4226828399 0 0) 0x7fab94003610 con 0x7fabc8157780 2026-02-25T15:08:00.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fabc81372e0 con 0x7fabc8157780 2026-02-25T15:08:00.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fabbc003310 con 0x7fabc8157780 2026-02-25T15:08:00.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:00.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:08:00.981 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 10 monclient(hunting): dump: 2026-02-25T15:08:00.981 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.981 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.981 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.981 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.981 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.981 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.981 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.982 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.982 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.982 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3587242260 0 0) 0x7fabc80a87c0 con 0x7fabc8164d00 2026-02-25T15:08:00.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fab94003610 con 0x7fabc8164d00 2026-02-25T15:08:00.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fabb8003330 con 0x7fabc8164d00 2026-02-25T15:08:00.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:00.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:08:00.982 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.966+0000 7fabaeffd640 10 monclient(hunting): dump: 2026-02-25T15:08:00.982 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:00.982 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:00.982 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.983 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:00.983 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:00.983 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:00.983 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:00.983 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:00.983 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:00.983 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:00.983 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1373274359 0 0) 0x7fabc80a8aa0 con 0x7fabc8157b50 2026-02-25T15:08:00.983 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 >> v1:10.20.193.94:6789/0 conn(0x7fabc8164d00 legacy=0x7fabc8147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.983 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 >> v1:10.20.193.90:6789/0 conn(0x7fabc8157780 legacy=0x7fabc80a7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:00.983 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabaeffd640 10 monclient: _finish_hunting 0 2026-02-25T15:08:00.983 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabaeffd640 1 monclient: found mon.b 2026-02-25T15:08:00.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabaeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:00.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabaeffd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc80a8d80 con 0x7fabc8157b50 2026-02-25T15:08:00.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabaeffd640 10 monclient: _finish_auth 0 2026-02-25T15:08:00.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabaeffd640 10 monclient: _check_auth_tickets 2026-02-25T15:08:00.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabaeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:00.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabcce04640 5 monclient: authenticate success, global_id 27601 2026-02-25T15:08:00.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabcce04640 10 monclient: _renew_subs 2026-02-25T15:08:00.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabcce04640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabcce04640 1 -- 10.20.193.90:0/1218847322 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fabc80a9060 con 0x7fabc8157b50 2026-02-25T15:08:00.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabcce04640 10 client.27601.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:08:00.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabcce04640 10 monclient: _renew_subs 2026-02-25T15:08:00.984 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabcce04640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.985 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabcce04640 1 -- 10.20.193.90:0/1218847322 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fabc80a9620 con 0x7fabc8157b50 2026-02-25T15:08:00.985 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.967+0000 7fabcce04640 1 librados: init done 2026-02-25T15:08:00.985 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.968+0000 7fabcce04640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:00.985 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.968+0000 7fabcce04640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:00.985 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.968+0000 7fabcce04640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:00.985 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:00.968+0000 7fabcce04640 1 -- 10.20.193.90:0/1218847322 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc8057480 con 0x7fabc8157b50 2026-02-25T15:08:01.060 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.058+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fabb0003700 con 0x7fabc8157b50 2026-02-25T15:08:01.060 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.058+0000 7fabaeffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:01.060 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.058+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7fabb0003a50 con 0x7fabc8157b50 2026-02-25T15:08:01.060 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.058+0000 7fabaeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:01.060 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.058+0000 7fabaeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:08:01.060 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.058+0000 7fabaeffd640 10 monclient: dump: 2026-02-25T15:08:01.061 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:01.061 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:01.061 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:01.061 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:01.061 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:01.061 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:01.061 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:01.061 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:01.061 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:01.061 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:01.061 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.058+0000 7fabc4d72640 4 set_mon_vals no callback set 2026-02-25T15:08:01.062 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.059+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7fabb001cf30 con 0x7fabc8157b50 2026-02-25T15:08:01.063 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.060+0000 7fabc6575640 10 monclient: get_auth_request con 0x7fab9407a090 auth_method 0 2026-02-25T15:08:01.063 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.060+0000 7fabaeffd640 10 client.27601.objecter ms_handle_connect 0x7fab9407a090 2026-02-25T15:08:01.063 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.060+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7fabb0094820 con 0x7fabc8157b50 2026-02-25T15:08:01.063 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.060+0000 7fabaeffd640 10 client.27601.objecter ms_dispatch2 0x7fabc8000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:08:01.063 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.060+0000 7fabaeffd640 3 client.27601.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:08:01.063 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.060+0000 7fabaeffd640 3 client.27601.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:08:01.063 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.060+0000 7fabaeffd640 20 client.27601.objecter dump_active .. 0 homeless 2026-02-25T15:08:01.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.062+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7fabb005e180 con 0x7fabc8157b50 2026-02-25T15:08:01.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.062+0000 7fabaeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:01.064 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.062+0000 7fabaeffd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:08:01.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.170+0000 7fabcce04640 1 -- 10.20.193.90:0/1218847322 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "15", "target": ["mon-mgr", ""]}) -- 0x7fabc80b4d10 con 0x7fab9407a090 2026-02-25T15:08:01.172 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.170+0000 7fabaeffd640 1 -- 10.20.193.90:0/1218847322 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 15 to deep-scrub) ==== 42+0+0 (unknown 337234985 0 0) 0x7fabc80b4d10 con 0x7fab9407a090 2026-02-25T15:08:01.172 INFO:teuthology.orchestra.run.trial090.stderr:instructed osd(s) 15 to deep-scrub 2026-02-25T15:08:01.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.172+0000 7fabcce04640 10 librados: watch_flush enter 2026-02-25T15:08:01.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.172+0000 7fabcce04640 10 librados: watch_flush exit 2026-02-25T15:08:01.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.172+0000 7fabcce04640 20 client.27601.objecter shutdown clearing up homeless session... 2026-02-25T15:08:01.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.172+0000 7fabcce04640 10 client.27601.objecter successfully canceled tick 2026-02-25T15:08:01.174 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.172+0000 7fabcce04640 1 -- 10.20.193.90:0/1218847322 >> v1:10.20.193.189:6800/3647375982 conn(0x7fab9407a090 legacy=0x7fab9409a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:01.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.172+0000 7fabcce04640 10 monclient: shutdown 2026-02-25T15:08:01.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.172+0000 7fabcce04640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:01.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.172+0000 7fabcce04640 1 -- 10.20.193.90:0/1218847322 >> v1:10.20.193.93:6789/0 conn(0x7fabc8157b50 legacy=0x7fabc8136bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:01.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.172+0000 7fabcce04640 1 -- 10.20.193.90:0/1218847322 shutdown_connections 2026-02-25T15:08:01.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.172+0000 7fabcce04640 1 -- 10.20.193.90:0/1218847322 >> 10.20.193.90:0/1218847322 conn(0x7fabc80b2ef0 msgr2=0x7fabc8138310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:01.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.172+0000 7fabcce04640 1 -- 10.20.193.90:0/1218847322 shutdown_connections 2026-02-25T15:08:01.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.172+0000 7fabcce04640 1 -- 10.20.193.90:0/1218847322 wait complete. 2026-02-25T15:08:01.175 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.172+0000 7fabcce04640 1 librados: shutdown 2026-02-25T15:08:01.183 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T15:08:01.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.250+0000 7f83c0eb1640 10 monclient: get_monmap_and_config 2026-02-25T15:08:01.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.250+0000 7f83c0eb1640 10 monclient: build_initial_monmap 2026-02-25T15:08:01.252 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.250+0000 7f83c0eb1640 10 monclient: monmap: 2026-02-25T15:08:01.252 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:01.252 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:01.252 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:08:01.251146+0000 2026-02-25T15:08:01.253 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:08:01.251146+0000 2026-02-25T15:08:01.253 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:01.253 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:01.253 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:01.253 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:01.253 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:01.253 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:01.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.250+0000 7f83c0eb1640 1 Processor -- start 2026-02-25T15:08:01.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.251+0000 7f83c0eb1640 1 -- start start 2026-02-25T15:08:01.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.251+0000 7f83c0eb1640 10 monclient: init 2026-02-25T15:08:01.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.251+0000 7f83c0eb1640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:01.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.251+0000 7f83c0eb1640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:08:01.256 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.251+0000 7f83c0eb1640 10 monclient(hunting): picked mon.noname-a con 0x7f83bc058da0 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:01.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.251+0000 7f83c0eb1640 10 monclient(hunting): picked mon.noname-b con 0x7f83bc0b0410 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:01.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.251+0000 7f83c0eb1640 10 monclient(hunting): picked mon.noname-c con 0x7f83bc1626a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:01.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.251+0000 7f83c0eb1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83bc1655c0 con 0x7f83bc058da0 2026-02-25T15:08:01.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.251+0000 7f83c0eb1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83bc0b12b0 con 0x7f83bc0b0410 2026-02-25T15:08:01.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.251+0000 7f83c0eb1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83bc0b1590 con 0x7f83bc1626a0 2026-02-25T15:08:01.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.251+0000 7f83c0eb1640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:01.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.251+0000 7f83c0eb1640 10 monclient(hunting): authenticate will time out at 3370.851699s 2026-02-25T15:08:01.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.252+0000 7f83ba575640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f83bc058da0 0x7f83bc1647a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:40166/0 (socket says 10.20.193.90:40166) 2026-02-25T15:08:01.257 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.252+0000 7f83ba575640 1 -- 10.20.193.90:0/2067091364 learned_addr learned my addr 10.20.193.90:0/2067091364 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:01.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.253+0000 7f83b9573640 1 -- 10.20.193.90:0/2067091364 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 592482511 0 0) 0x7f83bc0b1590 con 0x7f83bc1626a0 2026-02-25T15:08:01.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.253+0000 7f83b9573640 10 monclient(hunting): _negotiate 2026-02-25T15:08:01.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.253+0000 7f83b9573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:01.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.253+0000 7f83b9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:01.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.253+0000 7f83b9573640 10 monclient(hunting): my global_id is 28742 2026-02-25T15:08:01.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.253+0000 7f83b9573640 1 -- 10.20.193.90:0/2067091364 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83a8003610 con 0x7f83bc1626a0 2026-02-25T15:08:01.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.253+0000 7f83b9573640 1 -- 10.20.193.90:0/2067091364 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2133386738 0 0) 0x7f83bc1655c0 con 0x7f83bc058da0 2026-02-25T15:08:01.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.253+0000 7f83b9573640 10 monclient(hunting): _negotiate 2026-02-25T15:08:01.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.253+0000 7f83b9573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:01.258 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.253+0000 7f83b9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:01.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.253+0000 7f83b9573640 10 monclient(hunting): my global_id is 30975 2026-02-25T15:08:01.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.253+0000 7f83b9573640 1 -- 10.20.193.90:0/2067091364 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83bc0b1590 con 0x7f83bc058da0 2026-02-25T15:08:01.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.254+0000 7f83b9573640 1 -- 10.20.193.90:0/2067091364 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2055227513 0 0) 0x7f83bc0b12b0 con 0x7f83bc0b0410 2026-02-25T15:08:01.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.254+0000 7f83b9573640 10 monclient(hunting): _negotiate 2026-02-25T15:08:01.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.254+0000 7f83b9573640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:01.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.254+0000 7f83b9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:01.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.254+0000 7f83b9573640 10 monclient(hunting): my global_id is 27604 2026-02-25T15:08:01.259 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.254+0000 7f83b9573640 1 -- 10.20.193.90:0/2067091364 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83bc1655c0 con 0x7f83bc0b0410 2026-02-25T15:08:01.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83b9573640 1 -- 10.20.193.90:0/2067091364 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 837353828 0 0) 0x7f83bc0b1590 con 0x7f83bc058da0 2026-02-25T15:08:01.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83b9573640 1 -- 10.20.193.90:0/2067091364 >> v1:10.20.193.94:6789/0 conn(0x7f83bc1626a0 legacy=0x7f83bc1c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:01.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83b9573640 1 -- 10.20.193.90:0/2067091364 >> v1:10.20.193.93:6789/0 conn(0x7f83bc0b0410 legacy=0x7f83bc164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:01.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83b9573640 10 monclient: _finish_hunting 0 2026-02-25T15:08:01.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83b9573640 1 monclient: found mon.noname-a 2026-02-25T15:08:01.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83b9573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:01.267 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83b9573640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:08:01.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83b9573640 1 -- 10.20.193.90:0/2067091364 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc0b1870 con 0x7f83bc058da0 2026-02-25T15:08:01.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83b9573640 10 monclient: _finish_auth 0 2026-02-25T15:08:01.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83b9573640 10 monclient: _check_auth_tickets 2026-02-25T15:08:01.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83b9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:01.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83b9573640 1 -- 10.20.193.90:0/2067091364 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f83b0002df0 con 0x7f83bc058da0 2026-02-25T15:08:01.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83b9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:01.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83b9573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:08:01.268 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83b9573640 10 monclient: dump: 2026-02-25T15:08:01.268 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:01.268 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:01.268 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:01.268 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:01.269 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:01.269 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:01.269 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:01.269 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:01.269 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:01.269 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:01.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83c0eb1640 5 monclient: authenticate success, global_id 30975 2026-02-25T15:08:01.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.255+0000 7f83c0eb1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:08:01.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.256+0000 7f83b9573640 1 -- 10.20.193.90:0/2067091364 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f83b0002f90 con 0x7f83bc058da0 2026-02-25T15:08:01.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.256+0000 7f83b9573640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:01.269 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.256+0000 7f83b9573640 1 -- 10.20.193.90:0/2067091364 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f83b00033d0 con 0x7f83bc058da0 2026-02-25T15:08:01.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.256+0000 7f83b9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:01.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.256+0000 7f83b9573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:08:01.270 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.256+0000 7f83b9573640 10 monclient: dump: 2026-02-25T15:08:01.270 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:01.270 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:01.270 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:01.270 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:01.270 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:01.270 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:01.270 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:01.270 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:01.270 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:01.271 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:01.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.256+0000 7f83c0eb1640 10 monclient: get_monmap_and_config success 2026-02-25T15:08:01.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.256+0000 7f83c0eb1640 4 set_mon_vals no callback set 2026-02-25T15:08:01.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.256+0000 7f83c0eb1640 10 monclient: shutdown 2026-02-25T15:08:01.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.256+0000 7f83c0eb1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:01.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.256+0000 7f83c0eb1640 1 -- 10.20.193.90:0/2067091364 >> v1:10.20.193.90:6789/0 conn(0x7f83bc058da0 legacy=0x7f83bc1647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:01.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.256+0000 7f83c0eb1640 1 -- 10.20.193.90:0/2067091364 shutdown_connections 2026-02-25T15:08:01.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.256+0000 7f83c0eb1640 1 -- 10.20.193.90:0/2067091364 >> 10.20.193.90:0/2067091364 conn(0x7f83bc0b65f0 msgr2=0x7f83bc0b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:01.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.256+0000 7f83c0eb1640 1 -- 10.20.193.90:0/2067091364 shutdown_connections 2026-02-25T15:08:01.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.256+0000 7f83c0eb1640 1 -- 10.20.193.90:0/2067091364 wait complete. 2026-02-25T15:08:01.271 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 10 monclient: build_initial_monmap 2026-02-25T15:08:01.272 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 10 monclient: monmap: 2026-02-25T15:08:01.272 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:01.272 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:08:01.272 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:08:01.272 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:08:01.272 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:01.272 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:08:01.272 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:01.272 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:01.272 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:01.272 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:01.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 1 librados: starting msgr at 2026-02-25T15:08:01.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 1 librados: starting objecter 2026-02-25T15:08:01.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:08:01.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 1 Processor -- start 2026-02-25T15:08:01.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 1 -- start start 2026-02-25T15:08:01.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 1 librados: setting wanted keys 2026-02-25T15:08:01.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 1 librados: calling monclient init 2026-02-25T15:08:01.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 10 monclient: init 2026-02-25T15:08:01.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:01.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-25T15:08:01.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 10 monclient(hunting): picked mon.noname-a con 0x7f83bc058da0 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:01.273 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 10 monclient(hunting): picked mon.noname-b con 0x7f83bc0b0410 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:01.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 10 monclient(hunting): picked mon.noname-c con 0x7f83bc1626a0 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:01.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83bc1fc760 con 0x7f83bc058da0 2026-02-25T15:08:01.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83bc11aea0 con 0x7f83bc0b0410 2026-02-25T15:08:01.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83bc1609e0 con 0x7f83bc1626a0 2026-02-25T15:08:01.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:01.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.257+0000 7f83c0eb1640 10 monclient(hunting): authenticate will time out at 3370.857874s 2026-02-25T15:08:01.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.258+0000 7f83b9d74640 1 --1- >> v1:10.20.193.93:6789/0 conn(0x7f83bc0b0410 0x7f83bc1617a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.93:6789/0 says I am v1:10.20.193.90:46008/0 (socket says 10.20.193.90:46008) 2026-02-25T15:08:01.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.258+0000 7f83b9d74640 1 -- 10.20.193.90:0/1886630942 learned_addr learned my addr 10.20.193.90:0/1886630942 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:01.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.258+0000 7f83a2ffd640 10 client.?.objecter ms_handle_connect 0x7f83bc1626a0 2026-02-25T15:08:01.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.258+0000 7f83a2ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:01.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.258+0000 7f83a2ffd640 10 client.?.objecter ms_handle_connect 0x7f83bc0b0410 2026-02-25T15:08:01.274 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.258+0000 7f83a2ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:01.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.259+0000 7f83a2ffd640 10 client.?.objecter ms_handle_connect 0x7f83bc058da0 2026-02-25T15:08:01.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.259+0000 7f83a2ffd640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:01.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.259+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4143247113 0 0) 0x7f83bc11aea0 con 0x7f83bc0b0410 2026-02-25T15:08:01.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.259+0000 7f83a2ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:08:01.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.259+0000 7f83a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:01.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.259+0000 7f83a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:01.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.259+0000 7f83a2ffd640 10 monclient(hunting): my global_id is 27607 2026-02-25T15:08:01.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.259+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8388003610 con 0x7f83bc0b0410 2026-02-25T15:08:01.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.259+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3577913590 0 0) 0x7f83bc1609e0 con 0x7f83bc1626a0 2026-02-25T15:08:01.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.259+0000 7f83a2ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:08:01.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.259+0000 7f83a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:01.275 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.259+0000 7f83a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:01.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.259+0000 7f83a2ffd640 10 monclient(hunting): my global_id is 28745 2026-02-25T15:08:01.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.259+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83bc11aea0 con 0x7f83bc1626a0 2026-02-25T15:08:01.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.260+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 913657954 0 0) 0x7f83bc1fc760 con 0x7f83bc058da0 2026-02-25T15:08:01.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.260+0000 7f83a2ffd640 10 monclient(hunting): _negotiate 2026-02-25T15:08:01.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.260+0000 7f83a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:01.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.260+0000 7f83a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:01.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.260+0000 7f83a2ffd640 10 monclient(hunting): my global_id is 30978 2026-02-25T15:08:01.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.260+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83bc1609e0 con 0x7f83bc058da0 2026-02-25T15:08:01.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.261+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1263036251 0 0) 0x7f8388003610 con 0x7f83bc0b0410 2026-02-25T15:08:01.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.261+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f83bc1fc760 con 0x7f83bc0b0410 2026-02-25T15:08:01.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f83a4003170 con 0x7f83bc0b0410 2026-02-25T15:08:01.276 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:01.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-25T15:08:01.277 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 10 monclient(hunting): dump: 2026-02-25T15:08:01.277 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:01.277 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:01.277 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:01.277 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:01.277 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:01.277 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:01.277 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:01.277 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:01.277 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:01.277 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:01.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:08:01.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3596127017 0 0) 0x7f83bc11aea0 con 0x7f83bc1626a0 2026-02-25T15:08:01.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8388003610 con 0x7f83bc1626a0 2026-02-25T15:08:01.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f83ac003270 con 0x7f83bc1626a0 2026-02-25T15:08:01.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:01.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:08:01.278 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 10 monclient(hunting): dump: 2026-02-25T15:08:01.278 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:01.278 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:01.278 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:01.278 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:01.278 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:01.279 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:01.279 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:01.279 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:01.279 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:01.279 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:01.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 57907949 0 0) 0x7f83bc1609e0 con 0x7f83bc058da0 2026-02-25T15:08:01.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f83bc11aea0 con 0x7f83bc058da0 2026-02-25T15:08:01.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f83b00032e0 con 0x7f83bc058da0 2026-02-25T15:08:01.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:01.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:08:01.279 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.262+0000 7f83a2ffd640 10 monclient(hunting): dump: 2026-02-25T15:08:01.280 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:01.280 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:01.280 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:01.280 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:01.280 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:01.280 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:01.280 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:01.280 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:01.280 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:01.280 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:01.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mon.1 v1:10.20.193.93:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1647471954 0 0) 0x7f83bc1fc760 con 0x7f83bc0b0410 2026-02-25T15:08:01.280 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 >> v1:10.20.193.94:6789/0 conn(0x7f83bc1626a0 legacy=0x7f83bc135b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:01.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 >> v1:10.20.193.90:6789/0 conn(0x7f83bc058da0 legacy=0x7f83bc1fc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:01.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83a2ffd640 10 monclient: _finish_hunting 0 2026-02-25T15:08:01.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83a2ffd640 1 monclient: found mon.b 2026-02-25T15:08:01.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:01.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83a2ffd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:01.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 --> v1:10.20.193.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc160cc0 con 0x7f83bc0b0410 2026-02-25T15:08:01.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83a2ffd640 10 monclient: _finish_auth 0 2026-02-25T15:08:01.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83a2ffd640 10 monclient: _check_auth_tickets 2026-02-25T15:08:01.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:01.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83c0eb1640 5 monclient: authenticate success, global_id 27607 2026-02-25T15:08:01.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83c0eb1640 10 monclient: _renew_subs 2026-02-25T15:08:01.281 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83c0eb1640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:01.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83c0eb1640 1 -- 10.20.193.90:0/1886630942 --> v1:10.20.193.93:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f83bc160fa0 con 0x7f83bc0b0410 2026-02-25T15:08:01.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83c0eb1640 10 client.27607.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:08:01.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83c0eb1640 10 monclient: _renew_subs 2026-02-25T15:08:01.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83c0eb1640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:01.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83c0eb1640 1 -- 10.20.193.90:0/1886630942 --> v1:10.20.193.93:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f83bc161eb0 con 0x7f83bc0b0410 2026-02-25T15:08:01.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.263+0000 7f83c0eb1640 1 librados: init done 2026-02-25T15:08:01.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.264+0000 7f83c0eb1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:01.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.264+0000 7f83c0eb1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:01.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.264+0000 7f83c0eb1640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.93:6789/0 2026-02-25T15:08:01.282 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.264+0000 7f83c0eb1640 1 -- 10.20.193.90:0/1886630942 --> v1:10.20.193.93:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83bc0b28c0 con 0x7f83bc0b0410 2026-02-25T15:08:01.355 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.354+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mon.1 v1:10.20.193.93:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f83a4003520 con 0x7f83bc0b0410 2026-02-25T15:08:01.356 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.354+0000 7f83a2ffd640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:01.356 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.354+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mon.1 v1:10.20.193.93:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f83a4003b50 con 0x7f83bc0b0410 2026-02-25T15:08:01.356 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.354+0000 7f83a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:01.356 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.354+0000 7f83a2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-25T15:08:01.356 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.354+0000 7f83b8d72640 4 set_mon_vals no callback set 2026-02-25T15:08:01.356 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.354+0000 7f83a2ffd640 10 monclient: dump: 2026-02-25T15:08:01.356 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:01.356 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:01.356 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:01.356 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:01.357 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:01.357 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:01.357 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:01.357 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:01.357 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:01.357 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:01.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.354+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mon.1 v1:10.20.193.93:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f83a4003d30 con 0x7f83bc0b0410 2026-02-25T15:08:01.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.355+0000 7f83ba575640 10 monclient: get_auth_request con 0x7f838807a0f0 auth_method 0 2026-02-25T15:08:01.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.355+0000 7f83a2ffd640 10 client.27607.objecter ms_handle_connect 0x7f838807a0f0 2026-02-25T15:08:01.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.356+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mon.1 v1:10.20.193.93:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7f83a4095ad0 con 0x7f83bc0b0410 2026-02-25T15:08:01.358 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.356+0000 7f83a2ffd640 10 client.27607.objecter ms_dispatch2 0x7f83bc000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:08:01.359 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.356+0000 7f83a2ffd640 3 client.27607.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:08:01.359 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.356+0000 7f83a2ffd640 3 client.27607.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:08:01.359 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.356+0000 7f83a2ffd640 20 client.27607.objecter dump_active .. 0 homeless 2026-02-25T15:08:01.359 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.357+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mon.1 v1:10.20.193.93:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f83a405f430 con 0x7f83bc0b0410 2026-02-25T15:08:01.359 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.357+0000 7f83a2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:01.359 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.357+0000 7f83a2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:08:01.465 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.463+0000 7f83c0eb1640 1 -- 10.20.193.90:0/1886630942 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83bc162190 con 0x7f838807a0f0 2026-02-25T15:08:01.467 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.465+0000 7f83a2ffd640 1 -- 10.20.193.90:0/1886630942 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+125457 (unknown 2965378022 0 2681885580) 0x7f83bc162190 con 0x7f838807a0f0 2026-02-25T15:08:01.468 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:08:01.469 INFO:teuthology.orchestra.run.trial090.stderr:dumped all 2026-02-25T15:08:01.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.467+0000 7f83c0eb1640 10 librados: watch_flush enter 2026-02-25T15:08:01.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.467+0000 7f83c0eb1640 10 librados: watch_flush exit 2026-02-25T15:08:01.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.467+0000 7f83c0eb1640 20 client.27607.objecter shutdown clearing up homeless session... 2026-02-25T15:08:01.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.467+0000 7f83c0eb1640 10 client.27607.objecter successfully canceled tick 2026-02-25T15:08:01.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.467+0000 7f83c0eb1640 1 -- 10.20.193.90:0/1886630942 >> v1:10.20.193.189:6800/3647375982 conn(0x7f838807a0f0 legacy=0x7f838809a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:01.471 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.467+0000 7f83c0eb1640 10 monclient: shutdown 2026-02-25T15:08:01.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.467+0000 7f83c0eb1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:01.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.467+0000 7f83c0eb1640 1 -- 10.20.193.90:0/1886630942 >> v1:10.20.193.93:6789/0 conn(0x7f83bc0b0410 legacy=0x7f83bc1617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:01.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.467+0000 7f83c0eb1640 1 -- 10.20.193.90:0/1886630942 shutdown_connections 2026-02-25T15:08:01.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.467+0000 7f83c0eb1640 1 -- 10.20.193.90:0/1886630942 >> 10.20.193.90:0/1886630942 conn(0x7f83bc0b65f0 msgr2=0x7f83bc11b290 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:01.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.467+0000 7f83c0eb1640 1 -- 10.20.193.90:0/1886630942 shutdown_connections 2026-02-25T15:08:01.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.467+0000 7f83c0eb1640 1 -- 10.20.193.90:0/1886630942 wait complete. 2026-02-25T15:08:01.472 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:01.467+0000 7f83c0eb1640 1 librados: shutdown 2026-02-25T15:08:01.478 INFO:teuthology.orchestra.run.trial090.stdout:{"pg_ready":true,"pg_map":{"version":536,"stamp":"2026-02-25T15:08:01.141562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":56,"num_read_kb":47,"num_write":70,"num_write_kb":597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":557960,"kb_used_data":108552,"kb_used_omap":21857,"kb_used_meta":427294,"kb_avail":1509391480,"statfs":{"total":1546188226560,"available":1545616875520,"internally_reserved":0,"allocated":111157248,"data_stored":68934521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22382347,"internal_metadata":437549301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":124,"apply_latency_ms":124,"commit_latency_ns":124000000,"apply_latency_ns":124000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.548438"},"pg_stats":[{"pgid":"25.5","version":"0'0","reported_seq":627,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.906641+0000","last_change":"2026-02-25T15:02:14.833079+0000","last_active":"2026-02-25T15:07:56.906641+0000","last_peered":"2026-02-25T15:07:56.906641+0000","last_clean":"2026-02-25T15:07:56.906641+0000","last_became_active":"2026-02-25T15:02:14.830708+0000","last_became_peered":"2026-02-25T15:02:14.830708+0000","last_unstale":"2026-02-25T15:07:56.906641+0000","last_undegraded":"2026-02-25T15:07:56.906641+0000","last_fullsized":"2026-02-25T15:07:56.906641+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:21:07.436552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,8],"acting":[14,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":634,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:58.052213+0000","last_change":"2026-02-25T15:07:58.052213+0000","last_active":"2026-02-25T15:07:58.052213+0000","last_peered":"2026-02-25T15:07:58.052213+0000","last_clean":"2026-02-25T15:07:58.052213+0000","last_became_active":"2026-02-25T15:02:14.686130+0000","last_became_peered":"2026-02-25T15:02:14.686130+0000","last_unstale":"2026-02-25T15:07:58.052213+0000","last_undegraded":"2026-02-25T15:07:58.052213+0000","last_fullsized":"2026-02-25T15:07:58.052213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:58.047233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:58.047233+0000","last_clean_scrub_stamp":"2026-02-25T15:07:58.047233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:29:51.840515+0000","scrub_duration":122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7],"acting":[9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":621,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.437378+0000","last_change":"2026-02-25T15:02:14.184346+0000","last_active":"2026-02-25T15:07:56.437378+0000","last_peered":"2026-02-25T15:07:56.437378+0000","last_clean":"2026-02-25T15:07:56.437378+0000","last_became_active":"2026-02-25T15:02:14.182253+0000","last_became_peered":"2026-02-25T15:02:14.182253+0000","last_unstale":"2026-02-25T15:07:56.437378+0000","last_undegraded":"2026-02-25T15:07:56.437378+0000","last_fullsized":"2026-02-25T15:07:56.437378+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:10:33.826830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":621,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.436167+0000","last_change":"2026-02-25T15:02:14.262914+0000","last_active":"2026-02-25T15:07:56.436167+0000","last_peered":"2026-02-25T15:07:56.436167+0000","last_clean":"2026-02-25T15:07:56.436167+0000","last_became_active":"2026-02-25T15:02:14.258456+0000","last_became_peered":"2026-02-25T15:02:14.258456+0000","last_unstale":"2026-02-25T15:07:56.436167+0000","last_undegraded":"2026-02-25T15:07:56.436167+0000","last_fullsized":"2026-02-25T15:07:56.436167+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:29:58.464087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"338'12","reported_seq":660,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:59.415224+0000","last_change":"2026-02-25T15:07:59.415224+0000","last_active":"2026-02-25T15:07:59.415224+0000","last_peered":"2026-02-25T15:07:59.415224+0000","last_clean":"2026-02-25T15:07:59.415224+0000","last_became_active":"2026-02-25T15:02:15.819620+0000","last_became_peered":"2026-02-25T15:02:15.819620+0000","last_unstale":"2026-02-25T15:07:59.415224+0000","last_undegraded":"2026-02-25T15:07:59.415224+0000","last_fullsized":"2026-02-25T15:07:59.415224+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"338'12","last_scrub_stamp":"2026-02-25T15:07:59.411274+0000","last_deep_scrub":"338'12","last_deep_scrub_stamp":"2026-02-25T15:07:59.411274+0000","last_clean_scrub_stamp":"2026-02-25T15:07:59.411274+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:32:34.324039+0000","scrub_duration":160,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,13],"acting":[10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":630,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:57.034272+0000","last_change":"2026-02-25T15:07:56.065763+0000","last_active":"2026-02-25T15:07:57.034272+0000","last_peered":"2026-02-25T15:07:57.034272+0000","last_clean":"2026-02-25T15:07:57.034272+0000","last_became_active":"2026-02-25T15:02:13.773022+0000","last_became_peered":"2026-02-25T15:02:13.773022+0000","last_unstale":"2026-02-25T15:07:57.034272+0000","last_undegraded":"2026-02-25T15:07:57.034272+0000","last_fullsized":"2026-02-25T15:07:57.034272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:56.063893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:56.063893+0000","last_clean_scrub_stamp":"2026-02-25T15:07:56.063893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:03:01.213598+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":634,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:58.800820+0000","last_change":"2026-02-25T15:07:58.800820+0000","last_active":"2026-02-25T15:07:58.800820+0000","last_peered":"2026-02-25T15:07:58.800820+0000","last_clean":"2026-02-25T15:07:58.800820+0000","last_became_active":"2026-02-25T15:02:14.028952+0000","last_became_peered":"2026-02-25T15:02:14.028952+0000","last_unstale":"2026-02-25T15:07:58.800820+0000","last_undegraded":"2026-02-25T15:07:58.800820+0000","last_fullsized":"2026-02-25T15:07:58.800820+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:58.798610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:58.798610+0000","last_clean_scrub_stamp":"2026-02-25T15:07:58.798610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:22:29.742213+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,13],"acting":[5,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":621,"reported_epoch":337,"state":"active+clean","last_fresh":"2026-02-25T15:07:52.656835+0000","last_change":"2026-02-25T15:02:14.938617+0000","last_active":"2026-02-25T15:07:52.656835+0000","last_peered":"2026-02-25T15:07:52.656835+0000","last_clean":"2026-02-25T15:07:52.656835+0000","last_became_active":"2026-02-25T15:02:14.936242+0000","last_became_peered":"2026-02-25T15:02:14.936242+0000","last_unstale":"2026-02-25T15:07:52.656835+0000","last_undegraded":"2026-02-25T15:07:52.656835+0000","last_fullsized":"2026-02-25T15:07:52.656835+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:59:06.072509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":629,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:57.987432+0000","last_change":"2026-02-25T15:07:57.987432+0000","last_active":"2026-02-25T15:07:57.987432+0000","last_peered":"2026-02-25T15:07:57.987432+0000","last_clean":"2026-02-25T15:07:57.987432+0000","last_became_active":"2026-02-25T15:02:13.976916+0000","last_became_peered":"2026-02-25T15:02:13.976916+0000","last_unstale":"2026-02-25T15:07:57.987432+0000","last_undegraded":"2026-02-25T15:07:57.987432+0000","last_fullsized":"2026-02-25T15:07:57.987432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:57.982227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:57.982227+0000","last_clean_scrub_stamp":"2026-02-25T15:07:57.982227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:52:43.753406+0000","scrub_duration":89,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":630,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:57.036143+0000","last_change":"2026-02-25T15:07:57.027180+0000","last_active":"2026-02-25T15:07:57.036143+0000","last_peered":"2026-02-25T15:07:57.036143+0000","last_clean":"2026-02-25T15:07:57.036143+0000","last_became_active":"2026-02-25T15:02:13.382450+0000","last_became_peered":"2026-02-25T15:02:13.382450+0000","last_unstale":"2026-02-25T15:07:57.036143+0000","last_undegraded":"2026-02-25T15:07:57.036143+0000","last_fullsized":"2026-02-25T15:07:57.036143+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:57.024885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:57.024885+0000","last_clean_scrub_stamp":"2026-02-25T15:07:57.024885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:00:12.326161+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10],"acting":[0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":623,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.458585+0000","last_change":"2026-02-25T15:02:13.940723+0000","last_active":"2026-02-25T15:07:56.458585+0000","last_peered":"2026-02-25T15:07:56.458585+0000","last_clean":"2026-02-25T15:07:56.458585+0000","last_became_active":"2026-02-25T15:02:13.934323+0000","last_became_peered":"2026-02-25T15:02:13.934323+0000","last_unstale":"2026-02-25T15:07:56.458585+0000","last_undegraded":"2026-02-25T15:07:56.458585+0000","last_fullsized":"2026-02-25T15:07:56.458585+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:14:02.898606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10],"acting":[11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":634,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:59.754711+0000","last_change":"2026-02-25T15:07:59.754711+0000","last_active":"2026-02-25T15:07:59.754711+0000","last_peered":"2026-02-25T15:07:59.754711+0000","last_clean":"2026-02-25T15:07:59.754711+0000","last_became_active":"2026-02-25T15:02:14.063362+0000","last_became_peered":"2026-02-25T15:02:14.063362+0000","last_unstale":"2026-02-25T15:07:59.754711+0000","last_undegraded":"2026-02-25T15:07:59.754711+0000","last_fullsized":"2026-02-25T15:07:59.754711+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:59.753908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:59.753908+0000","last_clean_scrub_stamp":"2026-02-25T15:07:59.753908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:18:08.898103+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,14],"acting":[5,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":621,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.435116+0000","last_change":"2026-02-25T15:02:13.851333+0000","last_active":"2026-02-25T15:07:56.435116+0000","last_peered":"2026-02-25T15:07:56.435116+0000","last_clean":"2026-02-25T15:07:56.435116+0000","last_became_active":"2026-02-25T15:02:13.849038+0000","last_became_peered":"2026-02-25T15:02:13.849038+0000","last_unstale":"2026-02-25T15:07:56.435116+0000","last_undegraded":"2026-02-25T15:07:56.435116+0000","last_fullsized":"2026-02-25T15:07:56.435116+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:04:11.634095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":638,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.457849+0000","last_change":"2026-02-25T15:01:40.617020+0000","last_active":"2026-02-25T15:07:56.457849+0000","last_peered":"2026-02-25T15:07:56.457849+0000","last_clean":"2026-02-25T15:07:56.457849+0000","last_became_active":"2026-02-25T15:01:38.997857+0000","last_became_peered":"2026-02-25T15:01:38.997857+0000","last_unstale":"2026-02-25T15:07:56.457849+0000","last_undegraded":"2026-02-25T15:07:56.457849+0000","last_fullsized":"2026-02-25T15:07:56.457849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:49:56.506302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062975700999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14],"acting":[11,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":631,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:59.489817+0000","last_change":"2026-02-25T15:07:59.489817+0000","last_active":"2026-02-25T15:07:59.489817+0000","last_peered":"2026-02-25T15:07:59.489817+0000","last_clean":"2026-02-25T15:07:59.489817+0000","last_became_active":"2026-02-25T15:02:13.713599+0000","last_became_peered":"2026-02-25T15:02:13.713599+0000","last_unstale":"2026-02-25T15:07:59.489817+0000","last_undegraded":"2026-02-25T15:07:59.489817+0000","last_fullsized":"2026-02-25T15:07:59.489817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:59.488155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:59.488155+0000","last_clean_scrub_stamp":"2026-02-25T15:07:59.488155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:55:01.725878+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":648,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:57.005038+0000","last_change":"2026-02-25T15:07:57.005038+0000","last_active":"2026-02-25T15:07:57.005038+0000","last_peered":"2026-02-25T15:07:57.005038+0000","last_clean":"2026-02-25T15:07:57.005038+0000","last_became_active":"2026-02-25T15:01:38.782341+0000","last_became_peered":"2026-02-25T15:01:38.782341+0000","last_unstale":"2026-02-25T15:07:57.005038+0000","last_undegraded":"2026-02-25T15:07:57.005038+0000","last_fullsized":"2026-02-25T15:07:57.005038+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:57.003280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:57.003280+0000","last_clean_scrub_stamp":"2026-02-25T15:07:57.003280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:56:07.162968+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.031008292,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,3],"acting":[12,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":631,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:00.315326+0000","last_change":"2026-02-25T15:08:00.315326+0000","last_active":"2026-02-25T15:08:00.315326+0000","last_peered":"2026-02-25T15:08:00.315326+0000","last_clean":"2026-02-25T15:08:00.315326+0000","last_became_active":"2026-02-25T15:02:13.846214+0000","last_became_peered":"2026-02-25T15:02:13.846214+0000","last_unstale":"2026-02-25T15:08:00.315326+0000","last_undegraded":"2026-02-25T15:08:00.315326+0000","last_fullsized":"2026-02-25T15:08:00.315326+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:00.310619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:00.310619+0000","last_clean_scrub_stamp":"2026-02-25T15:08:00.310619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:09:16.071127+0000","scrub_duration":115,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11],"acting":[3,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":637,"reported_epoch":337,"state":"active+clean","last_fresh":"2026-02-25T15:07:52.660957+0000","last_change":"2026-02-25T15:01:41.143783+0000","last_active":"2026-02-25T15:07:52.660957+0000","last_peered":"2026-02-25T15:07:52.660957+0000","last_clean":"2026-02-25T15:07:52.660957+0000","last_became_active":"2026-02-25T15:01:38.731569+0000","last_became_peered":"2026-02-25T15:01:38.731569+0000","last_unstale":"2026-02-25T15:07:52.660957+0000","last_undegraded":"2026-02-25T15:07:52.660957+0000","last_fullsized":"2026-02-25T15:07:52.660957+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:00:25.099174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037093318,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15],"acting":[7,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":627,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.909669+0000","last_change":"2026-02-25T15:02:14.257499+0000","last_active":"2026-02-25T15:07:56.909669+0000","last_peered":"2026-02-25T15:07:56.909669+0000","last_clean":"2026-02-25T15:07:56.909669+0000","last_became_active":"2026-02-25T15:02:14.254843+0000","last_became_peered":"2026-02-25T15:02:14.254843+0000","last_unstale":"2026-02-25T15:07:56.909669+0000","last_undegraded":"2026-02-25T15:07:56.909669+0000","last_fullsized":"2026-02-25T15:07:56.909669+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:07:11.424289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2],"acting":[14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":637,"reported_epoch":337,"state":"active+clean","last_fresh":"2026-02-25T15:07:52.667328+0000","last_change":"2026-02-25T15:01:41.139794+0000","last_active":"2026-02-25T15:07:52.667328+0000","last_peered":"2026-02-25T15:07:52.667328+0000","last_clean":"2026-02-25T15:07:52.667328+0000","last_became_active":"2026-02-25T15:01:38.780556+0000","last_became_peered":"2026-02-25T15:01:38.780556+0000","last_unstale":"2026-02-25T15:07:52.667328+0000","last_undegraded":"2026-02-25T15:07:52.667328+0000","last_fullsized":"2026-02-25T15:07:52.667328+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:57:49.842137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041710525999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":630,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:58.087020+0000","last_change":"2026-02-25T15:07:58.087020+0000","last_active":"2026-02-25T15:07:58.087020+0000","last_peered":"2026-02-25T15:07:58.087020+0000","last_clean":"2026-02-25T15:07:58.087020+0000","last_became_active":"2026-02-25T15:02:14.894984+0000","last_became_peered":"2026-02-25T15:02:14.894984+0000","last_unstale":"2026-02-25T15:07:58.087020+0000","last_undegraded":"2026-02-25T15:07:58.087020+0000","last_fullsized":"2026-02-25T15:07:58.087020+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:58.085037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:58.085037+0000","last_clean_scrub_stamp":"2026-02-25T15:07:58.085037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:25:14.137958+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":641,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.454544+0000","last_change":"2026-02-25T15:01:40.608781+0000","last_active":"2026-02-25T15:07:56.454544+0000","last_peered":"2026-02-25T15:07:56.454544+0000","last_clean":"2026-02-25T15:07:56.454544+0000","last_became_active":"2026-02-25T15:01:38.743941+0000","last_became_peered":"2026-02-25T15:01:38.743941+0000","last_unstale":"2026-02-25T15:07:56.454544+0000","last_undegraded":"2026-02-25T15:07:56.454544+0000","last_fullsized":"2026-02-25T15:07:56.454544+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:53:32.920667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.061484652000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13],"acting":[11,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":634,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:57.959493+0000","last_change":"2026-02-25T15:07:57.959493+0000","last_active":"2026-02-25T15:07:57.959493+0000","last_peered":"2026-02-25T15:07:57.959493+0000","last_clean":"2026-02-25T15:07:57.959493+0000","last_became_active":"2026-02-25T15:02:14.364385+0000","last_became_peered":"2026-02-25T15:02:14.364385+0000","last_unstale":"2026-02-25T15:07:57.959493+0000","last_undegraded":"2026-02-25T15:07:57.959493+0000","last_fullsized":"2026-02-25T15:07:57.959493+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:57.954256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:57.954256+0000","last_clean_scrub_stamp":"2026-02-25T15:07:57.954256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:54:20.622843+0000","scrub_duration":124,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":647,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:58.566592+0000","last_change":"2026-02-25T15:07:58.566592+0000","last_active":"2026-02-25T15:07:58.566592+0000","last_peered":"2026-02-25T15:07:58.566592+0000","last_clean":"2026-02-25T15:07:58.566592+0000","last_became_active":"2026-02-25T15:01:38.861789+0000","last_became_peered":"2026-02-25T15:01:38.861789+0000","last_unstale":"2026-02-25T15:07:58.566592+0000","last_undegraded":"2026-02-25T15:07:58.566592+0000","last_fullsized":"2026-02-25T15:07:58.566592+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:58.561853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:58.561853+0000","last_clean_scrub_stamp":"2026-02-25T15:07:58.561853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:36:23.706461+0000","scrub_duration":95,"objects_trimmed":0,"snaptrim_duration":0.032448785000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":623,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.453273+0000","last_change":"2026-02-25T15:02:15.000767+0000","last_active":"2026-02-25T15:07:56.453273+0000","last_peered":"2026-02-25T15:07:56.453273+0000","last_clean":"2026-02-25T15:07:56.453273+0000","last_became_active":"2026-02-25T15:02:14.998340+0000","last_became_peered":"2026-02-25T15:02:14.998340+0000","last_unstale":"2026-02-25T15:07:56.453273+0000","last_undegraded":"2026-02-25T15:07:56.453273+0000","last_fullsized":"2026-02-25T15:07:56.453273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:06:28.044352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5],"acting":[11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":646,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:57.033521+0000","last_change":"2026-02-25T15:07:55.123074+0000","last_active":"2026-02-25T15:07:57.033521+0000","last_peered":"2026-02-25T15:07:57.033521+0000","last_clean":"2026-02-25T15:07:57.033521+0000","last_became_active":"2026-02-25T15:01:38.737942+0000","last_became_peered":"2026-02-25T15:01:38.737942+0000","last_unstale":"2026-02-25T15:07:57.033521+0000","last_undegraded":"2026-02-25T15:07:57.033521+0000","last_fullsized":"2026-02-25T15:07:57.033521+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:55.117759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:55.117759+0000","last_clean_scrub_stamp":"2026-02-25T15:07:55.117759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:25:01.007966+0000","scrub_duration":100,"objects_trimmed":0,"snaptrim_duration":0.031295004000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13],"acting":[0,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":621,"reported_epoch":337,"state":"active+clean","last_fresh":"2026-02-25T15:07:52.639052+0000","last_change":"2026-02-25T15:02:13.590713+0000","last_active":"2026-02-25T15:07:52.639052+0000","last_peered":"2026-02-25T15:07:52.639052+0000","last_clean":"2026-02-25T15:07:52.639052+0000","last_became_active":"2026-02-25T15:02:13.588299+0000","last_became_peered":"2026-02-25T15:02:13.588299+0000","last_unstale":"2026-02-25T15:07:52.639052+0000","last_undegraded":"2026-02-25T15:07:52.639052+0000","last_fullsized":"2026-02-25T15:07:52.639052+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:10:21.612478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":640,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.455995+0000","last_change":"2026-02-25T15:01:40.614625+0000","last_active":"2026-02-25T15:07:56.455995+0000","last_peered":"2026-02-25T15:07:56.455995+0000","last_clean":"2026-02-25T15:07:56.455995+0000","last_became_active":"2026-02-25T15:01:38.791923+0000","last_became_peered":"2026-02-25T15:01:38.791923+0000","last_unstale":"2026-02-25T15:07:56.455995+0000","last_undegraded":"2026-02-25T15:07:56.455995+0000","last_fullsized":"2026-02-25T15:07:56.455995+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:01:37.358375+0000","last_clean_scrub_stamp":"2026-02-25T15:01:37.358375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:51:10.001377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062725172999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5],"acting":[11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"1.0","version":"337'34","reported_seq":696,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.452140+0000","last_change":"2026-02-25T15:07:55.822179+0000","last_active":"2026-02-25T15:07:56.452140+0000","last_peered":"2026-02-25T15:07:56.452140+0000","last_clean":"2026-02-25T15:07:56.452140+0000","last_became_active":"2026-02-25T15:01:30.851092+0000","last_became_peered":"2026-02-25T15:01:30.851092+0000","last_unstale":"2026-02-25T15:07:56.452140+0000","last_undegraded":"2026-02-25T15:07:56.452140+0000","last_fullsized":"2026-02-25T15:07:56.452140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"337'34","last_scrub_stamp":"2026-02-25T15:07:55.817057+0000","last_deep_scrub":"337'34","last_deep_scrub_stamp":"2026-02-25T15:07:55.817057+0000","last_clean_scrub_stamp":"2026-02-25T15:07:55.817057+0000","objects_scrubbed":2,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:19:06.419651+0000","scrub_duration":173,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3],"acting":[8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":621,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.433784+0000","last_change":"2026-02-25T15:02:15.278232+0000","last_active":"2026-02-25T15:07:56.433784+0000","last_peered":"2026-02-25T15:07:56.433784+0000","last_clean":"2026-02-25T15:07:56.433784+0000","last_became_active":"2026-02-25T15:02:15.276113+0000","last_became_peered":"2026-02-25T15:02:15.276113+0000","last_unstale":"2026-02-25T15:07:56.433784+0000","last_undegraded":"2026-02-25T15:07:56.433784+0000","last_fullsized":"2026-02-25T15:07:56.433784+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:15:46.080630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":627,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.905210+0000","last_change":"2026-02-25T15:02:14.248177+0000","last_active":"2026-02-25T15:07:56.905210+0000","last_peered":"2026-02-25T15:07:56.905210+0000","last_clean":"2026-02-25T15:07:56.905210+0000","last_became_active":"2026-02-25T15:02:14.243187+0000","last_became_peered":"2026-02-25T15:02:14.243187+0000","last_unstale":"2026-02-25T15:07:56.905210+0000","last_undegraded":"2026-02-25T15:07:56.905210+0000","last_fullsized":"2026-02-25T15:07:56.905210+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:34:38.222587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,10],"acting":[14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":627,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.903533+0000","last_change":"2026-02-25T15:02:13.817879+0000","last_active":"2026-02-25T15:07:56.903533+0000","last_peered":"2026-02-25T15:07:56.903533+0000","last_clean":"2026-02-25T15:07:56.903533+0000","last_became_active":"2026-02-25T15:02:13.815222+0000","last_became_peered":"2026-02-25T15:02:13.815222+0000","last_unstale":"2026-02-25T15:07:56.903533+0000","last_undegraded":"2026-02-25T15:07:56.903533+0000","last_fullsized":"2026-02-25T15:07:56.903533+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:31:52.870792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,9],"acting":[14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":626,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.435273+0000","last_change":"2026-02-25T15:02:15.251677+0000","last_active":"2026-02-25T15:07:56.435273+0000","last_peered":"2026-02-25T15:07:56.435273+0000","last_clean":"2026-02-25T15:07:56.435273+0000","last_became_active":"2026-02-25T15:02:15.249084+0000","last_became_peered":"2026-02-25T15:02:15.249084+0000","last_unstale":"2026-02-25T15:07:56.435273+0000","last_undegraded":"2026-02-25T15:07:56.435273+0000","last_fullsized":"2026-02-25T15:07:56.435273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T02:14:00.305009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":633,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:58.001100+0000","last_change":"2026-02-25T15:07:58.001100+0000","last_active":"2026-02-25T15:07:58.001100+0000","last_peered":"2026-02-25T15:07:58.001100+0000","last_clean":"2026-02-25T15:07:58.001100+0000","last_became_active":"2026-02-25T15:02:14.836539+0000","last_became_peered":"2026-02-25T15:02:14.836539+0000","last_unstale":"2026-02-25T15:07:58.001100+0000","last_undegraded":"2026-02-25T15:07:58.001100+0000","last_fullsized":"2026-02-25T15:07:58.001100+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:57.998774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:57.998774+0000","last_clean_scrub_stamp":"2026-02-25T15:07:57.998774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:57:45.824759+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,13],"acting":[12,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":633,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:58.954156+0000","last_change":"2026-02-25T15:07:58.954156+0000","last_active":"2026-02-25T15:07:58.954156+0000","last_peered":"2026-02-25T15:07:58.954156+0000","last_clean":"2026-02-25T15:07:58.954156+0000","last_became_active":"2026-02-25T15:02:15.038261+0000","last_became_peered":"2026-02-25T15:02:15.038261+0000","last_unstale":"2026-02-25T15:07:58.954156+0000","last_undegraded":"2026-02-25T15:07:58.954156+0000","last_fullsized":"2026-02-25T15:07:58.954156+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:58.952439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:58.952439+0000","last_clean_scrub_stamp":"2026-02-25T15:07:58.952439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T02:44:48.219024+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,7],"acting":[12,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":633,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.737445+0000","last_change":"2026-02-25T15:07:56.737445+0000","last_active":"2026-02-25T15:07:56.737445+0000","last_peered":"2026-02-25T15:07:56.737445+0000","last_clean":"2026-02-25T15:07:56.737445+0000","last_became_active":"2026-02-25T15:02:13.426671+0000","last_became_peered":"2026-02-25T15:02:13.426671+0000","last_unstale":"2026-02-25T15:07:56.737445+0000","last_undegraded":"2026-02-25T15:07:56.737445+0000","last_fullsized":"2026-02-25T15:07:56.737445+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:56.735526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:56.735526+0000","last_clean_scrub_stamp":"2026-02-25T15:07:56.735526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:45:58.675259+0000","scrub_duration":57,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10],"acting":[8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":626,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.439514+0000","last_change":"2026-02-25T15:02:14.942239+0000","last_active":"2026-02-25T15:07:56.439514+0000","last_peered":"2026-02-25T15:07:56.439514+0000","last_clean":"2026-02-25T15:07:56.439514+0000","last_became_active":"2026-02-25T15:02:14.934430+0000","last_became_peered":"2026-02-25T15:02:14.934430+0000","last_unstale":"2026-02-25T15:07:56.439514+0000","last_undegraded":"2026-02-25T15:07:56.439514+0000","last_fullsized":"2026-02-25T15:07:56.439514+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:23:42.860020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8],"acting":[9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":633,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:59.943849+0000","last_change":"2026-02-25T15:07:59.943849+0000","last_active":"2026-02-25T15:07:59.943849+0000","last_peered":"2026-02-25T15:07:59.943849+0000","last_clean":"2026-02-25T15:07:59.943849+0000","last_became_active":"2026-02-25T15:02:14.164238+0000","last_became_peered":"2026-02-25T15:02:14.164238+0000","last_unstale":"2026-02-25T15:07:59.943849+0000","last_undegraded":"2026-02-25T15:07:59.943849+0000","last_fullsized":"2026-02-25T15:07:59.943849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:59.942708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:59.942708+0000","last_clean_scrub_stamp":"2026-02-25T15:07:59.942708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:58:39.617394+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,15],"acting":[12,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":634,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:59.938410+0000","last_change":"2026-02-25T15:07:59.938410+0000","last_active":"2026-02-25T15:07:59.938410+0000","last_peered":"2026-02-25T15:07:59.938410+0000","last_clean":"2026-02-25T15:07:59.938410+0000","last_became_active":"2026-02-25T15:02:15.078177+0000","last_became_peered":"2026-02-25T15:02:15.078177+0000","last_unstale":"2026-02-25T15:07:59.938410+0000","last_undegraded":"2026-02-25T15:07:59.938410+0000","last_fullsized":"2026-02-25T15:07:59.938410+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:59.937688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:59.937688+0000","last_clean_scrub_stamp":"2026-02-25T15:07:59.937688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:37:37.885433+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10],"acting":[9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":634,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:58.909817+0000","last_change":"2026-02-25T15:07:58.909817+0000","last_active":"2026-02-25T15:07:58.909817+0000","last_peered":"2026-02-25T15:07:58.909817+0000","last_clean":"2026-02-25T15:07:58.909817+0000","last_became_active":"2026-02-25T15:02:14.237082+0000","last_became_peered":"2026-02-25T15:02:14.237082+0000","last_unstale":"2026-02-25T15:07:58.909817+0000","last_undegraded":"2026-02-25T15:07:58.909817+0000","last_fullsized":"2026-02-25T15:07:58.909817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:58.907907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:58.907907+0000","last_clean_scrub_stamp":"2026-02-25T15:07:58.907907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:46:14.198318+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,14],"acting":[9,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":621,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.431916+0000","last_change":"2026-02-25T15:02:14.132167+0000","last_active":"2026-02-25T15:07:56.431916+0000","last_peered":"2026-02-25T15:07:56.431916+0000","last_clean":"2026-02-25T15:07:56.431916+0000","last_became_active":"2026-02-25T15:02:14.116657+0000","last_became_peered":"2026-02-25T15:02:14.116657+0000","last_unstale":"2026-02-25T15:07:56.431916+0000","last_undegraded":"2026-02-25T15:07:56.431916+0000","last_fullsized":"2026-02-25T15:07:56.431916+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:02:11.336161+0000","last_clean_scrub_stamp":"2026-02-25T15:02:11.336161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:23:36.296106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":2,"acting":2,"num_store_stats":2},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":16}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280593,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33696,"kb_used_data":6736,"kb_used_omap":5742,"kb_used_meta":21201,"kb_avail":94338144,"statfs":{"total":96636764160,"available":96602259456,"internally_reserved":0,"allocated":6897664,"data_stored":4251591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5880194,"internal_metadata":21710462},"hb_peers":[0,1,2,4,5,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280592,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33672,"kb_used_data":6712,"kb_used_omap":204,"kb_used_meta":26739,"kb_avail":94338168,"statfs":{"total":96636764160,"available":96602284032,"internally_reserved":0,"allocated":6873088,"data_stored":4242408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":209763,"internal_metadata":27380893},"hb_peers":[0,1,2,3,5,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280592,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34140,"kb_used_data":7180,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94337700,"statfs":{"total":96636764160,"available":96601804800,"internally_reserved":0,"allocated":7352320,"data_stored":4710847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210330,"internal_metadata":27380326},"hb_peers":[1,2,4,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313297,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51736,"kb_used_data":6728,"kb_used_omap":7598,"kb_used_meta":37393,"kb_avail":94320104,"statfs":{"total":96636764160,"available":96583786496,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7781354,"internal_metadata":38290454},"hb_peers":[0,2,3,4,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":14,"up_from":21,"seq":90194313296,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":208,"kb_used_meta":26735,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":214013,"internal_metadata":27376643},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313296,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210441,"internal_metadata":27380215},"hb_peers":[0,2,3,4,5,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313296,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":206,"kb_used_meta":26737,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":211892,"internal_metadata":27378764},"hb_peers":[1,2,3,4,5,6,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313296,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":207,"kb_used_meta":26736,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":212582,"internal_metadata":27378074},"hb_peers":[0,1,3,4,5,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313296,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210440,"internal_metadata":27380216},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313297,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210829,"internal_metadata":27379827},"hb_peers":[0,1,3,5,6,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313296,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":208,"kb_used_meta":26735,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":213501,"internal_metadata":27377155},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313297,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34140,"kb_used_data":7180,"kb_used_omap":208,"kb_used_meta":26735,"kb_avail":94337700,"statfs":{"total":96636764160,"available":96601804800,"internally_reserved":0,"allocated":7352320,"data_stored":4710847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":213221,"internal_metadata":27377435},"hb_peers":[0,2,3,4,5,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313296,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":207,"kb_used_meta":26736,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":212066,"internal_metadata":27378590},"hb_peers":[0,3,4,5,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313296,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33692,"kb_used_data":6732,"kb_used_omap":5828,"kb_used_meta":21115,"kb_avail":94338148,"statfs":{"total":96636764160,"available":96602263552,"internally_reserved":0,"allocated":6893568,"data_stored":4251572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5968665,"internal_metadata":21621991},"hb_peers":[0,2,3,4,5,6,8,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313297,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33692,"kb_used_data":6732,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338148,"statfs":{"total":96636764160,"available":96602263552,"internally_reserved":0,"allocated":6893568,"data_stored":4251586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210597,"internal_metadata":27380059},"hb_peers":[0,1,2,3,4,5,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313297,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":207,"kb_used_meta":26736,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":212459,"internal_metadata":27378197},"hb_peers":[1,3,4,5,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T15:08:01.485 INFO:tasks.ceph:pgid 25.5 last_scrub_stamp 2026-02-25T15:02:11.336161+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=2, tm_sec=11, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.486 INFO:tasks.ceph:pgid 25.7 last_scrub_stamp 2026-02-25T15:02:11.336161+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=2, tm_sec=11, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.486 INFO:tasks.ceph:pgid 25.6 last_scrub_stamp 2026-02-25T15:02:11.336161+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=2, tm_sec=11, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.486 INFO:tasks.ceph:pgid 25.2 last_scrub_stamp 2026-02-25T15:02:11.336161+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=2, tm_sec=11, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.486 INFO:tasks.ceph:pgid 25.f last_scrub_stamp 2026-02-25T15:02:11.336161+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=2, tm_sec=11, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.486 INFO:tasks.ceph:pgid 25.1c last_scrub_stamp 2026-02-25T15:02:11.336161+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=2, tm_sec=11, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.487 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-25T15:01:37.358375+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=1, tm_sec=37, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.487 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-25T15:01:37.358375+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=1, tm_sec=37, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.487 INFO:tasks.ceph:pgid 25.1f last_scrub_stamp 2026-02-25T15:02:11.336161+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=2, tm_sec=11, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.487 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-25T15:01:37.358375+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=1, tm_sec=37, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.487 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-25T15:01:37.358375+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=1, tm_sec=37, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.487 INFO:tasks.ceph:pgid 25.1b last_scrub_stamp 2026-02-25T15:02:11.336161+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=2, tm_sec=11, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.487 INFO:tasks.ceph:pgid 25.18 last_scrub_stamp 2026-02-25T15:02:11.336161+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=2, tm_sec=11, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.487 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-25T15:01:37.358375+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=1, tm_sec=37, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.488 INFO:tasks.ceph:pgid 25.12 last_scrub_stamp 2026-02-25T15:02:11.336161+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=2, tm_sec=11, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.488 INFO:tasks.ceph:pgid 25.13 last_scrub_stamp 2026-02-25T15:02:11.336161+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=2, tm_sec=11, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.488 INFO:tasks.ceph:pgid 25.10 last_scrub_stamp 2026-02-25T15:02:11.336161+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=2, tm_sec=11, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.488 INFO:tasks.ceph:pgid 25.11 last_scrub_stamp 2026-02-25T15:02:11.336161+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=2, tm_sec=11, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.488 INFO:tasks.ceph:pgid 25.15 last_scrub_stamp 2026-02-25T15:02:11.336161+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=2, tm_sec=11, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.488 INFO:tasks.ceph:pgid 25.9 last_scrub_stamp 2026-02-25T15:02:11.336161+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=2, tm_sec=11, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=15, tm_min=7, tm_sec=53, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T15:08:01.488 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-25T15:08:11.462 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:07:24.969 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:11.464 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:07:24.971 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:11.475 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:07:24.982 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:11.488 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:07:24.990 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:11.489 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:07:24.991 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:11.489 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:07:24.958 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:11.490 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:07:24.959 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:11.499 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:07:25.001 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:11.501 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:07:24.970 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:21.489 DEBUG:teuthology.orchestra.run.trial090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T15:08:21.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.556+0000 7f6085109640 10 monclient: get_monmap_and_config 2026-02-25T15:08:21.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.556+0000 7f6085109640 10 monclient: build_initial_monmap 2026-02-25T15:08:21.558 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.556+0000 7f6085109640 10 monclient: monmap: 2026-02-25T15:08:21.558 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:21.558 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:21.559 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:08:21.557312+0000 2026-02-25T15:08:21.559 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:08:21.557312+0000 2026-02-25T15:08:21.559 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:21.559 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:21.559 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:21.559 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:21.559 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:21.559 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.561 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.557+0000 7f6085109640 1 Processor -- start 2026-02-25T15:08:21.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.557+0000 7f6085109640 1 -- start start 2026-02-25T15:08:21.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.557+0000 7f6085109640 10 monclient: init 2026-02-25T15:08:21.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.557+0000 7f6085109640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:21.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.557+0000 7f6085109640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:08:21.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.557+0000 7f6085109640 10 monclient(hunting): picked mon.noname-c con 0x7f6080058da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:21.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.557+0000 7f6085109640 10 monclient(hunting): picked mon.noname-b con 0x7f60800b0410 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:21.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.557+0000 7f6085109640 10 monclient(hunting): picked mon.noname-a con 0x7f60801626a0 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:21.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.557+0000 7f6085109640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60801655c0 con 0x7f60801626a0 2026-02-25T15:08:21.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.557+0000 7f6085109640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60800b12b0 con 0x7f60800b0410 2026-02-25T15:08:21.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.557+0000 7f6085109640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60800b1590 con 0x7f6080058da0 2026-02-25T15:08:21.562 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.557+0000 7f6085109640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:21.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.557+0000 7f6085109640 10 monclient(hunting): authenticate will time out at 3391.157792s 2026-02-25T15:08:21.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.558+0000 7f607f577640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f60801626a0 0x7f60801c6d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:46084/0 (socket says 10.20.193.90:46084) 2026-02-25T15:08:21.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.558+0000 7f607f577640 1 -- 10.20.193.90:0/546514316 learned_addr learned my addr 10.20.193.90:0/546514316 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:21.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 1 -- 10.20.193.90:0/546514316 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3593518637 0 0) 0x7f60801655c0 con 0x7f60801626a0 2026-02-25T15:08:21.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 10 monclient(hunting): _negotiate 2026-02-25T15:08:21.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:21.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:21.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 10 monclient(hunting): my global_id is 30981 2026-02-25T15:08:21.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 1 -- 10.20.193.90:0/546514316 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f606c003610 con 0x7f60801626a0 2026-02-25T15:08:21.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 1 -- 10.20.193.90:0/546514316 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1771694976 0 0) 0x7f60800b12b0 con 0x7f60800b0410 2026-02-25T15:08:21.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 10 monclient(hunting): _negotiate 2026-02-25T15:08:21.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:21.563 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:21.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 10 monclient(hunting): my global_id is 27610 2026-02-25T15:08:21.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 1 -- 10.20.193.90:0/546514316 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60801655c0 con 0x7f60800b0410 2026-02-25T15:08:21.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 1 -- 10.20.193.90:0/546514316 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4237015021 0 0) 0x7f60800b1590 con 0x7f6080058da0 2026-02-25T15:08:21.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 10 monclient(hunting): _negotiate 2026-02-25T15:08:21.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:21.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:21.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 10 monclient(hunting): my global_id is 28748 2026-02-25T15:08:21.564 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.559+0000 7f607dd74640 1 -- 10.20.193.90:0/546514316 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60800b12b0 con 0x7f6080058da0 2026-02-25T15:08:21.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f607dd74640 1 -- 10.20.193.90:0/546514316 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 14827960 0 0) 0x7f606c003610 con 0x7f60801626a0 2026-02-25T15:08:21.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f607dd74640 1 -- 10.20.193.90:0/546514316 >> v1:10.20.193.94:6789/0 conn(0x7f6080058da0 legacy=0x7f60801647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:21.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f607dd74640 1 -- 10.20.193.90:0/546514316 >> v1:10.20.193.93:6789/0 conn(0x7f60800b0410 legacy=0x7f6080164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:21.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f607dd74640 10 monclient: _finish_hunting 0 2026-02-25T15:08:21.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f607dd74640 1 monclient: found mon.noname-a 2026-02-25T15:08:21.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f607dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:21.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f607dd74640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.90:6789/0 2026-02-25T15:08:21.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f607dd74640 1 -- 10.20.193.90:0/546514316 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60800b1870 con 0x7f60801626a0 2026-02-25T15:08:21.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f607dd74640 10 monclient: _finish_auth 0 2026-02-25T15:08:21.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f607dd74640 10 monclient: _check_auth_tickets 2026-02-25T15:08:21.570 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f607dd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:21.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f6085109640 5 monclient: authenticate success, global_id 30981 2026-02-25T15:08:21.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f607dd74640 1 -- 10.20.193.90:0/546514316 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6070003170 con 0x7f60801626a0 2026-02-25T15:08:21.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f6085109640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:08:21.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f607dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:21.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f607dd74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:08:21.571 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f607dd74640 10 monclient: dump: 2026-02-25T15:08:21.571 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:21.571 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:21.571 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.571 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.572 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:21.572 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:21.572 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:21.572 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:21.572 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:21.572 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.572 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.561+0000 7f6085109640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:08:21.572 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.562+0000 7f607dd74640 1 -- 10.20.193.90:0/546514316 <== mon.0 v1:10.20.193.90:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6070003340 con 0x7f60801626a0 2026-02-25T15:08:21.572 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.562+0000 7f607dd74640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:21.572 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.562+0000 7f607dd74640 1 -- 10.20.193.90:0/546514316 <== mon.0 v1:10.20.193.90:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6070003780 con 0x7f60801626a0 2026-02-25T15:08:21.572 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.562+0000 7f6085109640 10 monclient: get_monmap_and_config success 2026-02-25T15:08:21.572 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.562+0000 7f6085109640 4 set_mon_vals no callback set 2026-02-25T15:08:21.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.562+0000 7f607dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:21.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.562+0000 7f607dd74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:08:21.573 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.562+0000 7f607dd74640 10 monclient: dump: 2026-02-25T15:08:21.573 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:21.573 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:21.573 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.573 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.573 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:21.573 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:21.573 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:21.573 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:21.573 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:21.573 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.562+0000 7f6085109640 10 monclient: shutdown 2026-02-25T15:08:21.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.562+0000 7f6085109640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:21.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.562+0000 7f6085109640 1 -- 10.20.193.90:0/546514316 >> v1:10.20.193.90:6789/0 conn(0x7f60801626a0 legacy=0x7f60801c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:21.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.562+0000 7f6085109640 1 -- 10.20.193.90:0/546514316 shutdown_connections 2026-02-25T15:08:21.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.562+0000 7f6085109640 1 -- 10.20.193.90:0/546514316 >> 10.20.193.90:0/546514316 conn(0x7f60800b65f0 msgr2=0x7f60800b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:21.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.562+0000 7f6085109640 1 -- 10.20.193.90:0/546514316 shutdown_connections 2026-02-25T15:08:21.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.562+0000 7f6085109640 1 -- 10.20.193.90:0/546514316 wait complete. 2026-02-25T15:08:21.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 10 monclient: build_initial_monmap 2026-02-25T15:08:21.574 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 10 monclient: monmap: 2026-02-25T15:08:21.574 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:21.574 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:08:21.574 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:08:21.574 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:08:21.575 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:21.575 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:08:21.575 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:21.575 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:21.575 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:21.575 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 1 librados: starting msgr at 2026-02-25T15:08:21.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 1 librados: starting objecter 2026-02-25T15:08:21.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:08:21.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 1 Processor -- start 2026-02-25T15:08:21.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 1 -- start start 2026-02-25T15:08:21.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 1 librados: setting wanted keys 2026-02-25T15:08:21.575 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 1 librados: calling monclient init 2026-02-25T15:08:21.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 10 monclient: init 2026-02-25T15:08:21.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:21.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-25T15:08:21.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 10 monclient(hunting): picked mon.noname-c con 0x7f6080058da0 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:21.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 10 monclient(hunting): picked mon.noname-a con 0x7f60800b0410 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:21.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 10 monclient(hunting): picked mon.noname-b con 0x7f60801626a0 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:21.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6080138380 con 0x7f60800b0410 2026-02-25T15:08:21.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6080138660 con 0x7f60801626a0 2026-02-25T15:08:21.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6080138940 con 0x7f6080058da0 2026-02-25T15:08:21.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:21.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f6085109640 10 monclient(hunting): authenticate will time out at 3391.163762s 2026-02-25T15:08:21.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f607e575640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f60800b0410 0x7f60801327a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:46100/0 (socket says 10.20.193.90:46100) 2026-02-25T15:08:21.576 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.563+0000 7f607e575640 1 -- 10.20.193.90:0/578371006 learned_addr learned my addr 10.20.193.90:0/578371006 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:21.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.564+0000 7f605f7fe640 10 client.?.objecter ms_handle_connect 0x7f60800b0410 2026-02-25T15:08:21.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.564+0000 7f605f7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:21.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.564+0000 7f605f7fe640 10 client.?.objecter ms_handle_connect 0x7f60801626a0 2026-02-25T15:08:21.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.564+0000 7f605f7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:21.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.564+0000 7f605f7fe640 10 client.?.objecter ms_handle_connect 0x7f6080058da0 2026-02-25T15:08:21.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.564+0000 7f605f7fe640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:21.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 334631966 0 0) 0x7f6080138380 con 0x7f60800b0410 2026-02-25T15:08:21.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:08:21.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:21.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:21.577 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 10 monclient(hunting): my global_id is 30984 2026-02-25T15:08:21.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f604c003610 con 0x7f60800b0410 2026-02-25T15:08:21.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 646158621 0 0) 0x7f6080138940 con 0x7f6080058da0 2026-02-25T15:08:21.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:08:21.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:21.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:21.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 10 monclient(hunting): my global_id is 28751 2026-02-25T15:08:21.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6080138380 con 0x7f6080058da0 2026-02-25T15:08:21.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2139200867 0 0) 0x7f6080138660 con 0x7f60801626a0 2026-02-25T15:08:21.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 10 monclient(hunting): _negotiate 2026-02-25T15:08:21.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:21.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:21.578 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 10 monclient(hunting): my global_id is 27613 2026-02-25T15:08:21.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.565+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6080138940 con 0x7f60801626a0 2026-02-25T15:08:21.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1851099098 0 0) 0x7f604c003610 con 0x7f60800b0410 2026-02-25T15:08:21.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6080138660 con 0x7f60800b0410 2026-02-25T15:08:21.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6068003190 con 0x7f60800b0410 2026-02-25T15:08:21.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:21.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-25T15:08:21.579 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 10 monclient(hunting): dump: 2026-02-25T15:08:21.579 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:21.579 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:21.579 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.579 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.579 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:21.579 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:21.579 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:21.580 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:21.580 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:21.580 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:08:21.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 935273591 0 0) 0x7f6080138940 con 0x7f60801626a0 2026-02-25T15:08:21.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f604c003610 con 0x7f60801626a0 2026-02-25T15:08:21.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6070003690 con 0x7f60801626a0 2026-02-25T15:08:21.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:21.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:08:21.580 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 10 monclient(hunting): dump: 2026-02-25T15:08:21.580 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:21.580 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:21.581 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.581 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.581 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:21.581 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:21.581 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:21.581 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:21.581 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:21.581 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4201962799 0 0) 0x7f6080138380 con 0x7f6080058da0 2026-02-25T15:08:21.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6080138940 con 0x7f6080058da0 2026-02-25T15:08:21.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6074003060 con 0x7f6080058da0 2026-02-25T15:08:21.581 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:21.582 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-25T15:08:21.582 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.567+0000 7f605f7fe640 10 monclient(hunting): dump: 2026-02-25T15:08:21.582 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:21.582 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:21.582 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.582 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.582 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:21.582 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:21.582 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:21.582 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:21.582 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:21.582 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mon.0 v1:10.20.193.90:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 143876091 0 0) 0x7f6080138660 con 0x7f60800b0410 2026-02-25T15:08:21.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 >> v1:10.20.193.94:6789/0 conn(0x7f6080058da0 legacy=0x7f60801f9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:21.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 >> v1:10.20.193.93:6789/0 conn(0x7f60801626a0 legacy=0x7f60801fc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:21.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f605f7fe640 10 monclient: _finish_hunting 0 2026-02-25T15:08:21.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f605f7fe640 1 monclient: found mon.a 2026-02-25T15:08:21.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f605f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:21.585 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f605f7fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:08:21.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 --> v1:10.20.193.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6080138c20 con 0x7f60800b0410 2026-02-25T15:08:21.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f605f7fe640 10 monclient: _finish_auth 0 2026-02-25T15:08:21.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f605f7fe640 10 monclient: _check_auth_tickets 2026-02-25T15:08:21.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f605f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:21.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f6085109640 5 monclient: authenticate success, global_id 30984 2026-02-25T15:08:21.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f6085109640 10 monclient: _renew_subs 2026-02-25T15:08:21.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f6085109640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:08:21.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f6085109640 1 -- 10.20.193.90:0/578371006 --> v1:10.20.193.90:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6080138f00 con 0x7f60800b0410 2026-02-25T15:08:21.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f6085109640 10 client.30984.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:08:21.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f6085109640 10 monclient: _renew_subs 2026-02-25T15:08:21.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f6085109640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:08:21.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f6085109640 1 -- 10.20.193.90:0/578371006 --> v1:10.20.193.90:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f6080139700 con 0x7f60800b0410 2026-02-25T15:08:21.586 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f6085109640 1 librados: init done 2026-02-25T15:08:21.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f6085109640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:21.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f6085109640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:21.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f6085109640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.90:6789/0 2026-02-25T15:08:21.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.569+0000 7f6085109640 1 -- 10.20.193.90:0/578371006 --> v1:10.20.193.90:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6080163ee0 con 0x7f60800b0410 2026-02-25T15:08:21.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.577+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mon.0 v1:10.20.193.90:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f60680034e0 con 0x7f60800b0410 2026-02-25T15:08:21.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.577+0000 7f605f7fe640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:21.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.577+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mon.0 v1:10.20.193.90:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f6068003b50 con 0x7f60800b0410 2026-02-25T15:08:21.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.577+0000 7f605f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:21.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.577+0000 7f605f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-25T15:08:21.587 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.577+0000 7f605f7fe640 10 monclient: dump: 2026-02-25T15:08:21.587 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:21.587 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:21.587 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.588 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.588 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:21.588 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:21.588 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:21.588 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:21.588 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:21.588 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.588 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.577+0000 7f607d573640 4 set_mon_vals no callback set 2026-02-25T15:08:21.588 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.578+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mon.0 v1:10.20.193.90:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f6068003d30 con 0x7f60800b0410 2026-02-25T15:08:21.588 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.579+0000 7f607ed76640 10 monclient: get_auth_request con 0x7f604c079ff0 auth_method 0 2026-02-25T15:08:21.588 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.579+0000 7f605f7fe640 10 client.30984.objecter ms_handle_connect 0x7f604c079ff0 2026-02-25T15:08:21.588 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.580+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mon.0 v1:10.20.193.90:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7f6068095b00 con 0x7f60800b0410 2026-02-25T15:08:21.588 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.580+0000 7f605f7fe640 10 client.30984.objecter ms_dispatch2 0x7f6080000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:08:21.589 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.580+0000 7f605f7fe640 3 client.30984.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:08:21.589 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.580+0000 7f605f7fe640 3 client.30984.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:08:21.589 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.580+0000 7f605f7fe640 20 client.30984.objecter dump_active .. 0 homeless 2026-02-25T15:08:21.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.664+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mon.0 v1:10.20.193.90:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f6080163ee0 con 0x7f60800b0410 2026-02-25T15:08:21.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.664+0000 7f605f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:21.666 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.664+0000 7f605f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:08:21.770 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.768+0000 7f6085109640 1 -- 10.20.193.90:0/578371006 --> v1:10.20.193.189:6800/3647375982 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60800b6d40 con 0x7f604c079ff0 2026-02-25T15:08:21.772 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.770+0000 7f605f7fe640 1 -- 10.20.193.90:0/578371006 <== mgr.4107 v1:10.20.193.189:6800/3647375982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+125252 (unknown 2965378022 0 3142236946) 0x7f60800b6d40 con 0x7f604c079ff0 2026-02-25T15:08:21.773 INFO:teuthology.orchestra.run.trial090.stdout: 2026-02-25T15:08:21.774 INFO:teuthology.orchestra.run.trial090.stderr:dumped all 2026-02-25T15:08:21.775 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.771+0000 7f6085109640 10 librados: watch_flush enter 2026-02-25T15:08:21.775 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.771+0000 7f6085109640 10 librados: watch_flush exit 2026-02-25T15:08:21.775 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.771+0000 7f6085109640 20 client.30984.objecter shutdown clearing up homeless session... 2026-02-25T15:08:21.776 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.771+0000 7f6085109640 10 client.30984.objecter successfully canceled tick 2026-02-25T15:08:21.776 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.771+0000 7f6085109640 1 -- 10.20.193.90:0/578371006 >> v1:10.20.193.189:6800/3647375982 conn(0x7f604c079ff0 legacy=0x7f604c09a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:21.776 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.771+0000 7f6085109640 10 monclient: shutdown 2026-02-25T15:08:21.776 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.771+0000 7f6085109640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:21.776 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.771+0000 7f6085109640 1 -- 10.20.193.90:0/578371006 >> v1:10.20.193.90:6789/0 conn(0x7f60800b0410 legacy=0x7f60801327a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:21.776 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.771+0000 7f6085109640 1 -- 10.20.193.90:0/578371006 shutdown_connections 2026-02-25T15:08:21.776 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.771+0000 7f6085109640 1 -- 10.20.193.90:0/578371006 >> 10.20.193.90:0/578371006 conn(0x7f60800b65f0 msgr2=0x7f6080055b30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:21.776 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.771+0000 7f6085109640 1 -- 10.20.193.90:0/578371006 shutdown_connections 2026-02-25T15:08:21.776 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.771+0000 7f6085109640 1 -- 10.20.193.90:0/578371006 wait complete. 2026-02-25T15:08:21.776 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.771+0000 7f6085109640 1 librados: shutdown 2026-02-25T15:08:21.781 INFO:teuthology.orchestra.run.trial090.stdout:{"pg_ready":true,"pg_map":{"version":546,"stamp":"2026-02-25T15:08:21.145160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":56,"num_read_kb":47,"num_write":70,"num_write_kb":597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":557976,"kb_used_data":108568,"kb_used_omap":21858,"kb_used_meta":427293,"kb_avail":1509391464,"statfs":{"total":1546188226560,"available":1545616859136,"internally_reserved":0,"allocated":111173632,"data_stored":68943680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22382983,"internal_metadata":437548665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002140"},"pg_stats":[{"pgid":"25.5","version":"0'0","reported_seq":635,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:59.924414+0000","last_change":"2026-02-25T15:07:59.924414+0000","last_active":"2026-02-25T15:07:59.924414+0000","last_peered":"2026-02-25T15:07:59.924414+0000","last_clean":"2026-02-25T15:07:59.924414+0000","last_became_active":"2026-02-25T15:02:14.830708+0000","last_became_peered":"2026-02-25T15:02:14.830708+0000","last_unstale":"2026-02-25T15:07:59.924414+0000","last_undegraded":"2026-02-25T15:07:59.924414+0000","last_fullsized":"2026-02-25T15:07:59.924414+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:59.922658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:59.922658+0000","last_clean_scrub_stamp":"2026-02-25T15:07:59.922658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:25:23.200517+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,8],"acting":[14,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":634,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:58.052213+0000","last_change":"2026-02-25T15:07:58.052213+0000","last_active":"2026-02-25T15:07:58.052213+0000","last_peered":"2026-02-25T15:07:58.052213+0000","last_clean":"2026-02-25T15:07:58.052213+0000","last_became_active":"2026-02-25T15:02:14.686130+0000","last_became_peered":"2026-02-25T15:02:14.686130+0000","last_unstale":"2026-02-25T15:07:58.052213+0000","last_undegraded":"2026-02-25T15:07:58.052213+0000","last_fullsized":"2026-02-25T15:07:58.052213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:58.047233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:58.047233+0000","last_clean_scrub_stamp":"2026-02-25T15:07:58.047233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:29:51.840515+0000","scrub_duration":122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7],"acting":[9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":629,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:59.583710+0000","last_change":"2026-02-25T15:07:59.583710+0000","last_active":"2026-02-25T15:07:59.583710+0000","last_peered":"2026-02-25T15:07:59.583710+0000","last_clean":"2026-02-25T15:07:59.583710+0000","last_became_active":"2026-02-25T15:02:14.182253+0000","last_became_peered":"2026-02-25T15:02:14.182253+0000","last_unstale":"2026-02-25T15:07:59.583710+0000","last_undegraded":"2026-02-25T15:07:59.583710+0000","last_fullsized":"2026-02-25T15:07:59.583710+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:59.582666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:59.582666+0000","last_clean_scrub_stamp":"2026-02-25T15:07:59.582666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:25:11.894370+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":629,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:02.634803+0000","last_change":"2026-02-25T15:08:02.634803+0000","last_active":"2026-02-25T15:08:02.634803+0000","last_peered":"2026-02-25T15:08:02.634803+0000","last_clean":"2026-02-25T15:08:02.634803+0000","last_became_active":"2026-02-25T15:02:14.258456+0000","last_became_peered":"2026-02-25T15:02:14.258456+0000","last_unstale":"2026-02-25T15:08:02.634803+0000","last_undegraded":"2026-02-25T15:08:02.634803+0000","last_fullsized":"2026-02-25T15:08:02.634803+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:02.634002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:02.634002+0000","last_clean_scrub_stamp":"2026-02-25T15:08:02.634002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:29:42.553373+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"338'12","reported_seq":660,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:59.415224+0000","last_change":"2026-02-25T15:07:59.415224+0000","last_active":"2026-02-25T15:07:59.415224+0000","last_peered":"2026-02-25T15:07:59.415224+0000","last_clean":"2026-02-25T15:07:59.415224+0000","last_became_active":"2026-02-25T15:02:15.819620+0000","last_became_peered":"2026-02-25T15:02:15.819620+0000","last_unstale":"2026-02-25T15:07:59.415224+0000","last_undegraded":"2026-02-25T15:07:59.415224+0000","last_fullsized":"2026-02-25T15:07:59.415224+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"338'12","last_scrub_stamp":"2026-02-25T15:07:59.411274+0000","last_deep_scrub":"338'12","last_deep_scrub_stamp":"2026-02-25T15:07:59.411274+0000","last_clean_scrub_stamp":"2026-02-25T15:07:59.411274+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:32:34.324039+0000","scrub_duration":160,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,13],"acting":[10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":631,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:57.034591+0000","last_change":"2026-02-25T15:07:56.065763+0000","last_active":"2026-02-25T15:07:57.034591+0000","last_peered":"2026-02-25T15:07:57.034591+0000","last_clean":"2026-02-25T15:07:57.034591+0000","last_became_active":"2026-02-25T15:02:13.773022+0000","last_became_peered":"2026-02-25T15:02:13.773022+0000","last_unstale":"2026-02-25T15:07:57.034591+0000","last_undegraded":"2026-02-25T15:07:57.034591+0000","last_fullsized":"2026-02-25T15:07:57.034591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:56.063893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:56.063893+0000","last_clean_scrub_stamp":"2026-02-25T15:07:56.063893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:03:01.213598+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":634,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:58.800820+0000","last_change":"2026-02-25T15:07:58.800820+0000","last_active":"2026-02-25T15:07:58.800820+0000","last_peered":"2026-02-25T15:07:58.800820+0000","last_clean":"2026-02-25T15:07:58.800820+0000","last_became_active":"2026-02-25T15:02:14.028952+0000","last_became_peered":"2026-02-25T15:02:14.028952+0000","last_unstale":"2026-02-25T15:07:58.800820+0000","last_undegraded":"2026-02-25T15:07:58.800820+0000","last_fullsized":"2026-02-25T15:07:58.800820+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:58.798610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:58.798610+0000","last_clean_scrub_stamp":"2026-02-25T15:07:58.798610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:22:29.742213+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,13],"acting":[5,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":631,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:02.407836+0000","last_change":"2026-02-25T15:08:02.407836+0000","last_active":"2026-02-25T15:08:02.407836+0000","last_peered":"2026-02-25T15:08:02.407836+0000","last_clean":"2026-02-25T15:08:02.407836+0000","last_became_active":"2026-02-25T15:02:14.936242+0000","last_became_peered":"2026-02-25T15:02:14.936242+0000","last_unstale":"2026-02-25T15:08:02.407836+0000","last_undegraded":"2026-02-25T15:08:02.407836+0000","last_fullsized":"2026-02-25T15:08:02.407836+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:02.406137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:02.406137+0000","last_clean_scrub_stamp":"2026-02-25T15:08:02.406137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:52:42.461790+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":629,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:57.987432+0000","last_change":"2026-02-25T15:07:57.987432+0000","last_active":"2026-02-25T15:07:57.987432+0000","last_peered":"2026-02-25T15:07:57.987432+0000","last_clean":"2026-02-25T15:07:57.987432+0000","last_became_active":"2026-02-25T15:02:13.976916+0000","last_became_peered":"2026-02-25T15:02:13.976916+0000","last_unstale":"2026-02-25T15:07:57.987432+0000","last_undegraded":"2026-02-25T15:07:57.987432+0000","last_fullsized":"2026-02-25T15:07:57.987432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:57.982227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:57.982227+0000","last_clean_scrub_stamp":"2026-02-25T15:07:57.982227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:52:43.753406+0000","scrub_duration":89,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":630,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:57.036143+0000","last_change":"2026-02-25T15:07:57.027180+0000","last_active":"2026-02-25T15:07:57.036143+0000","last_peered":"2026-02-25T15:07:57.036143+0000","last_clean":"2026-02-25T15:07:57.036143+0000","last_became_active":"2026-02-25T15:02:13.382450+0000","last_became_peered":"2026-02-25T15:02:13.382450+0000","last_unstale":"2026-02-25T15:07:57.036143+0000","last_undegraded":"2026-02-25T15:07:57.036143+0000","last_fullsized":"2026-02-25T15:07:57.036143+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:57.024885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:57.024885+0000","last_clean_scrub_stamp":"2026-02-25T15:07:57.024885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:00:12.326161+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10],"acting":[0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":631,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:05.545392+0000","last_change":"2026-02-25T15:08:05.545392+0000","last_active":"2026-02-25T15:08:05.545392+0000","last_peered":"2026-02-25T15:08:05.545392+0000","last_clean":"2026-02-25T15:08:05.545392+0000","last_became_active":"2026-02-25T15:02:13.934323+0000","last_became_peered":"2026-02-25T15:02:13.934323+0000","last_unstale":"2026-02-25T15:08:05.545392+0000","last_undegraded":"2026-02-25T15:08:05.545392+0000","last_fullsized":"2026-02-25T15:08:05.545392+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:05.543605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:05.543605+0000","last_clean_scrub_stamp":"2026-02-25T15:08:05.543605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:43:20.771798+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10],"acting":[11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":634,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:59.754711+0000","last_change":"2026-02-25T15:07:59.754711+0000","last_active":"2026-02-25T15:07:59.754711+0000","last_peered":"2026-02-25T15:07:59.754711+0000","last_clean":"2026-02-25T15:07:59.754711+0000","last_became_active":"2026-02-25T15:02:14.063362+0000","last_became_peered":"2026-02-25T15:02:14.063362+0000","last_unstale":"2026-02-25T15:07:59.754711+0000","last_undegraded":"2026-02-25T15:07:59.754711+0000","last_fullsized":"2026-02-25T15:07:59.754711+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:59.753908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:59.753908+0000","last_clean_scrub_stamp":"2026-02-25T15:07:59.753908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:18:08.898103+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,14],"acting":[5,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":629,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:03.669867+0000","last_change":"2026-02-25T15:08:03.669867+0000","last_active":"2026-02-25T15:08:03.669867+0000","last_peered":"2026-02-25T15:08:03.669867+0000","last_clean":"2026-02-25T15:08:03.669867+0000","last_became_active":"2026-02-25T15:02:13.849038+0000","last_became_peered":"2026-02-25T15:02:13.849038+0000","last_unstale":"2026-02-25T15:08:03.669867+0000","last_undegraded":"2026-02-25T15:08:03.669867+0000","last_fullsized":"2026-02-25T15:08:03.669867+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:03.667804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:03.667804+0000","last_clean_scrub_stamp":"2026-02-25T15:08:03.667804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:47:51.920849+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":647,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:04.520436+0000","last_change":"2026-02-25T15:08:04.519784+0000","last_active":"2026-02-25T15:08:04.520436+0000","last_peered":"2026-02-25T15:08:04.520436+0000","last_clean":"2026-02-25T15:08:04.520436+0000","last_became_active":"2026-02-25T15:01:38.997857+0000","last_became_peered":"2026-02-25T15:01:38.997857+0000","last_unstale":"2026-02-25T15:08:04.520436+0000","last_undegraded":"2026-02-25T15:08:04.520436+0000","last_fullsized":"2026-02-25T15:08:04.520436+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:04.517699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:04.517699+0000","last_clean_scrub_stamp":"2026-02-25T15:08:04.517699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:05:56.540518+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.062975700999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14],"acting":[11,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":631,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:59.489817+0000","last_change":"2026-02-25T15:07:59.489817+0000","last_active":"2026-02-25T15:07:59.489817+0000","last_peered":"2026-02-25T15:07:59.489817+0000","last_clean":"2026-02-25T15:07:59.489817+0000","last_became_active":"2026-02-25T15:02:13.713599+0000","last_became_peered":"2026-02-25T15:02:13.713599+0000","last_unstale":"2026-02-25T15:07:59.489817+0000","last_undegraded":"2026-02-25T15:07:59.489817+0000","last_fullsized":"2026-02-25T15:07:59.489817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:59.488155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:59.488155+0000","last_clean_scrub_stamp":"2026-02-25T15:07:59.488155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:55:01.725878+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":649,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:57.005917+0000","last_change":"2026-02-25T15:07:57.005038+0000","last_active":"2026-02-25T15:07:57.005917+0000","last_peered":"2026-02-25T15:07:57.005917+0000","last_clean":"2026-02-25T15:07:57.005917+0000","last_became_active":"2026-02-25T15:01:38.782341+0000","last_became_peered":"2026-02-25T15:01:38.782341+0000","last_unstale":"2026-02-25T15:07:57.005917+0000","last_undegraded":"2026-02-25T15:07:57.005917+0000","last_fullsized":"2026-02-25T15:07:57.005917+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:57.003280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:57.003280+0000","last_clean_scrub_stamp":"2026-02-25T15:07:57.003280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:56:07.162968+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.031008292,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,3],"acting":[12,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":632,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:00.316127+0000","last_change":"2026-02-25T15:08:00.315326+0000","last_active":"2026-02-25T15:08:00.316127+0000","last_peered":"2026-02-25T15:08:00.316127+0000","last_clean":"2026-02-25T15:08:00.316127+0000","last_became_active":"2026-02-25T15:02:13.846214+0000","last_became_peered":"2026-02-25T15:02:13.846214+0000","last_unstale":"2026-02-25T15:08:00.316127+0000","last_undegraded":"2026-02-25T15:08:00.316127+0000","last_fullsized":"2026-02-25T15:08:00.316127+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:00.310619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:00.310619+0000","last_clean_scrub_stamp":"2026-02-25T15:08:00.310619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:09:16.071127+0000","scrub_duration":115,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11],"acting":[3,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":647,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:00.432264+0000","last_change":"2026-02-25T15:08:00.432264+0000","last_active":"2026-02-25T15:08:00.432264+0000","last_peered":"2026-02-25T15:08:00.432264+0000","last_clean":"2026-02-25T15:08:00.432264+0000","last_became_active":"2026-02-25T15:01:38.731569+0000","last_became_peered":"2026-02-25T15:01:38.731569+0000","last_unstale":"2026-02-25T15:08:00.432264+0000","last_undegraded":"2026-02-25T15:08:00.432264+0000","last_fullsized":"2026-02-25T15:08:00.432264+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:00.430637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:00.430637+0000","last_clean_scrub_stamp":"2026-02-25T15:08:00.430637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:37:53.110544+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.037093318,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15],"acting":[7,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":635,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:02.924247+0000","last_change":"2026-02-25T15:08:02.924247+0000","last_active":"2026-02-25T15:08:02.924247+0000","last_peered":"2026-02-25T15:08:02.924247+0000","last_clean":"2026-02-25T15:08:02.924247+0000","last_became_active":"2026-02-25T15:02:14.254843+0000","last_became_peered":"2026-02-25T15:02:14.254843+0000","last_unstale":"2026-02-25T15:08:02.924247+0000","last_undegraded":"2026-02-25T15:08:02.924247+0000","last_fullsized":"2026-02-25T15:08:02.924247+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:02.923485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:02.923485+0000","last_clean_scrub_stamp":"2026-02-25T15:08:02.923485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:18:30.109095+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2],"acting":[14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":647,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:01.427471+0000","last_change":"2026-02-25T15:08:01.427471+0000","last_active":"2026-02-25T15:08:01.427471+0000","last_peered":"2026-02-25T15:08:01.427471+0000","last_clean":"2026-02-25T15:08:01.427471+0000","last_became_active":"2026-02-25T15:01:38.780556+0000","last_became_peered":"2026-02-25T15:01:38.780556+0000","last_unstale":"2026-02-25T15:08:01.427471+0000","last_undegraded":"2026-02-25T15:08:01.427471+0000","last_fullsized":"2026-02-25T15:08:01.427471+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:01.426459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:01.426459+0000","last_clean_scrub_stamp":"2026-02-25T15:08:01.426459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:56:44.977472+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.041710525999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":630,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:58.087020+0000","last_change":"2026-02-25T15:07:58.087020+0000","last_active":"2026-02-25T15:07:58.087020+0000","last_peered":"2026-02-25T15:07:58.087020+0000","last_clean":"2026-02-25T15:07:58.087020+0000","last_became_active":"2026-02-25T15:02:14.894984+0000","last_became_peered":"2026-02-25T15:02:14.894984+0000","last_unstale":"2026-02-25T15:07:58.087020+0000","last_undegraded":"2026-02-25T15:07:58.087020+0000","last_fullsized":"2026-02-25T15:07:58.087020+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:58.085037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:58.085037+0000","last_clean_scrub_stamp":"2026-02-25T15:07:58.085037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:25:14.137958+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"338'3","reported_seq":650,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:01.520352+0000","last_change":"2026-02-25T15:08:01.520352+0000","last_active":"2026-02-25T15:08:01.520352+0000","last_peered":"2026-02-25T15:08:01.520352+0000","last_clean":"2026-02-25T15:08:01.520352+0000","last_became_active":"2026-02-25T15:01:38.743941+0000","last_became_peered":"2026-02-25T15:01:38.743941+0000","last_unstale":"2026-02-25T15:08:01.520352+0000","last_undegraded":"2026-02-25T15:08:01.520352+0000","last_fullsized":"2026-02-25T15:08:01.520352+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"338'3","last_scrub_stamp":"2026-02-25T15:08:01.519453+0000","last_deep_scrub":"338'3","last_deep_scrub_stamp":"2026-02-25T15:08:01.519453+0000","last_clean_scrub_stamp":"2026-02-25T15:08:01.519453+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:35:52.520967+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0.061484652000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13],"acting":[11,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":634,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:57.959493+0000","last_change":"2026-02-25T15:07:57.959493+0000","last_active":"2026-02-25T15:07:57.959493+0000","last_peered":"2026-02-25T15:07:57.959493+0000","last_clean":"2026-02-25T15:07:57.959493+0000","last_became_active":"2026-02-25T15:02:14.364385+0000","last_became_peered":"2026-02-25T15:02:14.364385+0000","last_unstale":"2026-02-25T15:07:57.959493+0000","last_undegraded":"2026-02-25T15:07:57.959493+0000","last_fullsized":"2026-02-25T15:07:57.959493+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:57.954256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:57.954256+0000","last_clean_scrub_stamp":"2026-02-25T15:07:57.954256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:54:20.622843+0000","scrub_duration":124,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":647,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:58.566592+0000","last_change":"2026-02-25T15:07:58.566592+0000","last_active":"2026-02-25T15:07:58.566592+0000","last_peered":"2026-02-25T15:07:58.566592+0000","last_clean":"2026-02-25T15:07:58.566592+0000","last_became_active":"2026-02-25T15:01:38.861789+0000","last_became_peered":"2026-02-25T15:01:38.861789+0000","last_unstale":"2026-02-25T15:07:58.566592+0000","last_undegraded":"2026-02-25T15:07:58.566592+0000","last_fullsized":"2026-02-25T15:07:58.566592+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:58.561853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:58.561853+0000","last_clean_scrub_stamp":"2026-02-25T15:07:58.561853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:36:23.706461+0000","scrub_duration":95,"objects_trimmed":0,"snaptrim_duration":0.032448785000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":631,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:03.496394+0000","last_change":"2026-02-25T15:08:03.496394+0000","last_active":"2026-02-25T15:08:03.496394+0000","last_peered":"2026-02-25T15:08:03.496394+0000","last_clean":"2026-02-25T15:08:03.496394+0000","last_became_active":"2026-02-25T15:02:14.998340+0000","last_became_peered":"2026-02-25T15:02:14.998340+0000","last_unstale":"2026-02-25T15:08:03.496394+0000","last_undegraded":"2026-02-25T15:08:03.496394+0000","last_fullsized":"2026-02-25T15:08:03.496394+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:03.495521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:03.495521+0000","last_clean_scrub_stamp":"2026-02-25T15:08:03.495521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:50:49.192167+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5],"acting":[11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":646,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:57.033521+0000","last_change":"2026-02-25T15:07:55.123074+0000","last_active":"2026-02-25T15:07:57.033521+0000","last_peered":"2026-02-25T15:07:57.033521+0000","last_clean":"2026-02-25T15:07:57.033521+0000","last_became_active":"2026-02-25T15:01:38.737942+0000","last_became_peered":"2026-02-25T15:01:38.737942+0000","last_unstale":"2026-02-25T15:07:57.033521+0000","last_undegraded":"2026-02-25T15:07:57.033521+0000","last_fullsized":"2026-02-25T15:07:57.033521+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:55.117759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:55.117759+0000","last_clean_scrub_stamp":"2026-02-25T15:07:55.117759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:25:01.007966+0000","scrub_duration":100,"objects_trimmed":0,"snaptrim_duration":0.031295004000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13],"acting":[0,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":631,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:03.451245+0000","last_change":"2026-02-25T15:08:03.451245+0000","last_active":"2026-02-25T15:08:03.451245+0000","last_peered":"2026-02-25T15:08:03.451245+0000","last_clean":"2026-02-25T15:08:03.451245+0000","last_became_active":"2026-02-25T15:02:13.588299+0000","last_became_peered":"2026-02-25T15:02:13.588299+0000","last_unstale":"2026-02-25T15:08:03.451245+0000","last_undegraded":"2026-02-25T15:08:03.451245+0000","last_fullsized":"2026-02-25T15:08:03.451245+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:03.449578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:03.449578+0000","last_clean_scrub_stamp":"2026-02-25T15:08:03.449578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:14:37.057145+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"338'2","reported_seq":649,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:02.508745+0000","last_change":"2026-02-25T15:08:02.508745+0000","last_active":"2026-02-25T15:08:02.508745+0000","last_peered":"2026-02-25T15:08:02.508745+0000","last_clean":"2026-02-25T15:08:02.508745+0000","last_became_active":"2026-02-25T15:01:38.791923+0000","last_became_peered":"2026-02-25T15:01:38.791923+0000","last_unstale":"2026-02-25T15:08:02.508745+0000","last_undegraded":"2026-02-25T15:08:02.508745+0000","last_fullsized":"2026-02-25T15:08:02.508745+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"338'2","last_scrub_stamp":"2026-02-25T15:08:02.507727+0000","last_deep_scrub":"338'2","last_deep_scrub_stamp":"2026-02-25T15:08:02.507727+0000","last_clean_scrub_stamp":"2026-02-25T15:08:02.507727+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:14:34.908088+0000","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0.062725172999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5],"acting":[11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"1.0","version":"337'34","reported_seq":696,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.452140+0000","last_change":"2026-02-25T15:07:55.822179+0000","last_active":"2026-02-25T15:07:56.452140+0000","last_peered":"2026-02-25T15:07:56.452140+0000","last_clean":"2026-02-25T15:07:56.452140+0000","last_became_active":"2026-02-25T15:01:30.851092+0000","last_became_peered":"2026-02-25T15:01:30.851092+0000","last_unstale":"2026-02-25T15:07:56.452140+0000","last_undegraded":"2026-02-25T15:07:56.452140+0000","last_fullsized":"2026-02-25T15:07:56.452140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"337'34","last_scrub_stamp":"2026-02-25T15:07:55.817057+0000","last_deep_scrub":"337'34","last_deep_scrub_stamp":"2026-02-25T15:07:55.817057+0000","last_clean_scrub_stamp":"2026-02-25T15:07:55.817057+0000","objects_scrubbed":2,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:19:06.419651+0000","scrub_duration":173,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3],"acting":[8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":629,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:01.626025+0000","last_change":"2026-02-25T15:08:01.626025+0000","last_active":"2026-02-25T15:08:01.626025+0000","last_peered":"2026-02-25T15:08:01.626025+0000","last_clean":"2026-02-25T15:08:01.626025+0000","last_became_active":"2026-02-25T15:02:15.276113+0000","last_became_peered":"2026-02-25T15:02:15.276113+0000","last_unstale":"2026-02-25T15:08:01.626025+0000","last_undegraded":"2026-02-25T15:08:01.626025+0000","last_fullsized":"2026-02-25T15:08:01.626025+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:01.624137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:01.624137+0000","last_clean_scrub_stamp":"2026-02-25T15:08:01.624137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:16:04.387234+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":635,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:00.883952+0000","last_change":"2026-02-25T15:08:00.883952+0000","last_active":"2026-02-25T15:08:00.883952+0000","last_peered":"2026-02-25T15:08:00.883952+0000","last_clean":"2026-02-25T15:08:00.883952+0000","last_became_active":"2026-02-25T15:02:14.243187+0000","last_became_peered":"2026-02-25T15:02:14.243187+0000","last_unstale":"2026-02-25T15:08:00.883952+0000","last_undegraded":"2026-02-25T15:08:00.883952+0000","last_fullsized":"2026-02-25T15:08:00.883952+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:00.882320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:00.882320+0000","last_clean_scrub_stamp":"2026-02-25T15:08:00.882320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:39:41.913369+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,10],"acting":[14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":635,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:01.881392+0000","last_change":"2026-02-25T15:08:01.881392+0000","last_active":"2026-02-25T15:08:01.881392+0000","last_peered":"2026-02-25T15:08:01.881392+0000","last_clean":"2026-02-25T15:08:01.881392+0000","last_became_active":"2026-02-25T15:02:13.815222+0000","last_became_peered":"2026-02-25T15:02:13.815222+0000","last_unstale":"2026-02-25T15:08:01.881392+0000","last_undegraded":"2026-02-25T15:08:01.881392+0000","last_fullsized":"2026-02-25T15:08:01.881392+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:01.880620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:01.880620+0000","last_clean_scrub_stamp":"2026-02-25T15:08:01.880620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:32:39.259811+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,9],"acting":[14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":634,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:00.982557+0000","last_change":"2026-02-25T15:08:00.982557+0000","last_active":"2026-02-25T15:08:00.982557+0000","last_peered":"2026-02-25T15:08:00.982557+0000","last_clean":"2026-02-25T15:08:00.982557+0000","last_became_active":"2026-02-25T15:02:15.249084+0000","last_became_peered":"2026-02-25T15:02:15.249084+0000","last_unstale":"2026-02-25T15:08:00.982557+0000","last_undegraded":"2026-02-25T15:08:00.982557+0000","last_fullsized":"2026-02-25T15:08:00.982557+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:00.980952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:00.980952+0000","last_clean_scrub_stamp":"2026-02-25T15:08:00.980952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T02:44:08.017089+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":633,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:58.001100+0000","last_change":"2026-02-25T15:07:58.001100+0000","last_active":"2026-02-25T15:07:58.001100+0000","last_peered":"2026-02-25T15:07:58.001100+0000","last_clean":"2026-02-25T15:07:58.001100+0000","last_became_active":"2026-02-25T15:02:14.836539+0000","last_became_peered":"2026-02-25T15:02:14.836539+0000","last_unstale":"2026-02-25T15:07:58.001100+0000","last_undegraded":"2026-02-25T15:07:58.001100+0000","last_fullsized":"2026-02-25T15:07:58.001100+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:57.998774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:57.998774+0000","last_clean_scrub_stamp":"2026-02-25T15:07:57.998774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:57:45.824759+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,13],"acting":[12,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":633,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:58.954156+0000","last_change":"2026-02-25T15:07:58.954156+0000","last_active":"2026-02-25T15:07:58.954156+0000","last_peered":"2026-02-25T15:07:58.954156+0000","last_clean":"2026-02-25T15:07:58.954156+0000","last_became_active":"2026-02-25T15:02:15.038261+0000","last_became_peered":"2026-02-25T15:02:15.038261+0000","last_unstale":"2026-02-25T15:07:58.954156+0000","last_undegraded":"2026-02-25T15:07:58.954156+0000","last_fullsized":"2026-02-25T15:07:58.954156+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:58.952439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:58.952439+0000","last_clean_scrub_stamp":"2026-02-25T15:07:58.952439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T02:44:48.219024+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,7],"acting":[12,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":633,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:56.737445+0000","last_change":"2026-02-25T15:07:56.737445+0000","last_active":"2026-02-25T15:07:56.737445+0000","last_peered":"2026-02-25T15:07:56.737445+0000","last_clean":"2026-02-25T15:07:56.737445+0000","last_became_active":"2026-02-25T15:02:13.426671+0000","last_became_peered":"2026-02-25T15:02:13.426671+0000","last_unstale":"2026-02-25T15:07:56.737445+0000","last_undegraded":"2026-02-25T15:07:56.737445+0000","last_fullsized":"2026-02-25T15:07:56.737445+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:56.735526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:56.735526+0000","last_clean_scrub_stamp":"2026-02-25T15:07:56.735526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:45:58.675259+0000","scrub_duration":57,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10],"acting":[8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":634,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:01.935105+0000","last_change":"2026-02-25T15:08:01.935105+0000","last_active":"2026-02-25T15:08:01.935105+0000","last_peered":"2026-02-25T15:08:01.935105+0000","last_clean":"2026-02-25T15:08:01.935105+0000","last_became_active":"2026-02-25T15:02:14.934430+0000","last_became_peered":"2026-02-25T15:02:14.934430+0000","last_unstale":"2026-02-25T15:08:01.935105+0000","last_undegraded":"2026-02-25T15:08:01.935105+0000","last_fullsized":"2026-02-25T15:08:01.935105+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:01.933633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:01.933633+0000","last_clean_scrub_stamp":"2026-02-25T15:08:01.933633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:33:17.358451+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8],"acting":[9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":633,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:59.943849+0000","last_change":"2026-02-25T15:07:59.943849+0000","last_active":"2026-02-25T15:07:59.943849+0000","last_peered":"2026-02-25T15:07:59.943849+0000","last_clean":"2026-02-25T15:07:59.943849+0000","last_became_active":"2026-02-25T15:02:14.164238+0000","last_became_peered":"2026-02-25T15:02:14.164238+0000","last_unstale":"2026-02-25T15:07:59.943849+0000","last_undegraded":"2026-02-25T15:07:59.943849+0000","last_fullsized":"2026-02-25T15:07:59.943849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:59.942708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:59.942708+0000","last_clean_scrub_stamp":"2026-02-25T15:07:59.942708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:58:39.617394+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,15],"acting":[12,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":634,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:59.938410+0000","last_change":"2026-02-25T15:07:59.938410+0000","last_active":"2026-02-25T15:07:59.938410+0000","last_peered":"2026-02-25T15:07:59.938410+0000","last_clean":"2026-02-25T15:07:59.938410+0000","last_became_active":"2026-02-25T15:02:15.078177+0000","last_became_peered":"2026-02-25T15:02:15.078177+0000","last_unstale":"2026-02-25T15:07:59.938410+0000","last_undegraded":"2026-02-25T15:07:59.938410+0000","last_fullsized":"2026-02-25T15:07:59.938410+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:59.937688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:59.937688+0000","last_clean_scrub_stamp":"2026-02-25T15:07:59.937688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T00:37:37.885433+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10],"acting":[9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":634,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:07:58.909817+0000","last_change":"2026-02-25T15:07:58.909817+0000","last_active":"2026-02-25T15:07:58.909817+0000","last_peered":"2026-02-25T15:07:58.909817+0000","last_clean":"2026-02-25T15:07:58.909817+0000","last_became_active":"2026-02-25T15:02:14.237082+0000","last_became_peered":"2026-02-25T15:02:14.237082+0000","last_unstale":"2026-02-25T15:07:58.909817+0000","last_undegraded":"2026-02-25T15:07:58.909817+0000","last_fullsized":"2026-02-25T15:07:58.909817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:07:58.907907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:07:58.907907+0000","last_clean_scrub_stamp":"2026-02-25T15:07:58.907907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-27T01:46:14.198318+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,14],"acting":[9,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":629,"reported_epoch":338,"state":"active+clean","last_fresh":"2026-02-25T15:08:00.585533+0000","last_change":"2026-02-25T15:08:00.585533+0000","last_active":"2026-02-25T15:08:00.585533+0000","last_peered":"2026-02-25T15:08:00.585533+0000","last_clean":"2026-02-25T15:08:00.585533+0000","last_became_active":"2026-02-25T15:02:14.116657+0000","last_became_peered":"2026-02-25T15:02:14.116657+0000","last_unstale":"2026-02-25T15:08:00.585533+0000","last_undegraded":"2026-02-25T15:08:00.585533+0000","last_fullsized":"2026-02-25T15:08:00.585533+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T15:08:00.583639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T15:08:00.583639+0000","last_clean_scrub_stamp":"2026-02-25T15:08:00.583639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:11:09.582260+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":2,"acting":2,"num_store_stats":2},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":16}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280597,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33696,"kb_used_data":6736,"kb_used_omap":5742,"kb_used_meta":21201,"kb_avail":94338144,"statfs":{"total":96636764160,"available":96602259456,"internally_reserved":0,"allocated":6897664,"data_stored":4251591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5880194,"internal_metadata":21710462},"hb_peers":[0,1,2,4,5,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280596,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210399,"internal_metadata":27380257},"hb_peers":[0,1,2,3,5,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280596,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34140,"kb_used_data":7180,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94337700,"statfs":{"total":96636764160,"available":96601804800,"internally_reserved":0,"allocated":7352320,"data_stored":4710847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210330,"internal_metadata":27380326},"hb_peers":[1,2,4,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313301,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51736,"kb_used_data":6728,"kb_used_omap":7598,"kb_used_meta":37393,"kb_avail":94320104,"statfs":{"total":96636764160,"available":96583786496,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7781354,"internal_metadata":38290454},"hb_peers":[0,2,3,4,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":21,"seq":90194313300,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":208,"kb_used_meta":26735,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":214013,"internal_metadata":27376643},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313300,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210441,"internal_metadata":27380215},"hb_peers":[0,2,3,4,5,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313300,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":206,"kb_used_meta":26737,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":211892,"internal_metadata":27378764},"hb_peers":[1,2,3,4,5,6,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313300,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":207,"kb_used_meta":26736,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":212582,"internal_metadata":27378074},"hb_peers":[0,1,3,4,5,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313300,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210440,"internal_metadata":27380216},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313301,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210829,"internal_metadata":27379827},"hb_peers":[0,1,3,5,6,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313300,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":208,"kb_used_meta":26735,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":213501,"internal_metadata":27377155},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313301,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34140,"kb_used_data":7180,"kb_used_omap":208,"kb_used_meta":26735,"kb_avail":94337700,"statfs":{"total":96636764160,"available":96601804800,"internally_reserved":0,"allocated":7352320,"data_stored":4710847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":213221,"internal_metadata":27377435},"hb_peers":[0,2,3,4,5,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313300,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":207,"kb_used_meta":26736,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":212066,"internal_metadata":27378590},"hb_peers":[0,3,4,5,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313300,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33692,"kb_used_data":6732,"kb_used_omap":5828,"kb_used_meta":21115,"kb_avail":94338148,"statfs":{"total":96636764160,"available":96602263552,"internally_reserved":0,"allocated":6893568,"data_stored":4251572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5968665,"internal_metadata":21621991},"hb_peers":[0,2,3,4,5,6,8,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313301,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33692,"kb_used_data":6732,"kb_used_omap":205,"kb_used_meta":26738,"kb_avail":94338148,"statfs":{"total":96636764160,"available":96602263552,"internally_reserved":0,"allocated":6893568,"data_stored":4251586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":210597,"internal_metadata":27380059},"hb_peers":[0,1,2,3,4,5,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313301,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":6728,"kb_used_omap":207,"kb_used_meta":26736,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":6889472,"data_stored":4251567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":212459,"internal_metadata":27378197},"hb_peers":[1,3,4,5,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T15:08:21.787 DEBUG:teuthology.orchestra.run.trial090:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-25T15:08:21.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.852+0000 7f948b399640 10 monclient: get_monmap_and_config 2026-02-25T15:08:21.853 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.852+0000 7f948b399640 10 monclient: build_initial_monmap 2026-02-25T15:08:21.854 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.852+0000 7f948b399640 10 monclient: monmap: 2026-02-25T15:08:21.854 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:21.854 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:21.854 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:08:21.852664+0000 2026-02-25T15:08:21.854 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:08:21.852664+0000 2026-02-25T15:08:21.854 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:21.854 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:21.854 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:21.854 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:21.855 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:21.855 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.852+0000 7f948b399640 1 Processor -- start 2026-02-25T15:08:21.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.852+0000 7f948b399640 1 -- start start 2026-02-25T15:08:21.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.852+0000 7f948b399640 10 monclient: init 2026-02-25T15:08:21.857 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.853+0000 7f948b399640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:21.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.853+0000 7f948b399640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-25T15:08:21.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.853+0000 7f948b399640 10 monclient(hunting): picked mon.noname-c con 0x7f948415b130 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:21.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.853+0000 7f948b399640 10 monclient(hunting): picked mon.noname-b con 0x7f94840b5d30 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:21.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.853+0000 7f948b399640 10 monclient(hunting): picked mon.noname-a con 0x7f94840b29d0 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:21.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.853+0000 7f948b399640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f948405dd20 con 0x7f94840b29d0 2026-02-25T15:08:21.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.853+0000 7f948b399640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94840b7ec0 con 0x7f94840b5d30 2026-02-25T15:08:21.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.853+0000 7f948b399640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94840b81a0 con 0x7f948415b130 2026-02-25T15:08:21.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.853+0000 7f948b399640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:21.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.853+0000 7f948b399640 10 monclient(hunting): authenticate will time out at 3391.453415s 2026-02-25T15:08:21.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.853+0000 7f948990f640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f94840b29d0 0x7f948405a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:46110/0 (socket says 10.20.193.90:46110) 2026-02-25T15:08:21.858 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.853+0000 7f948990f640 1 -- 10.20.193.90:0/2464761894 learned_addr learned my addr 10.20.193.90:0/2464761894 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:21.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.854+0000 7f9473fff640 1 -- 10.20.193.90:0/2464761894 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2588830558 0 0) 0x7f94840b81a0 con 0x7f948415b130 2026-02-25T15:08:21.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.854+0000 7f9473fff640 10 monclient(hunting): _negotiate 2026-02-25T15:08:21.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.854+0000 7f9473fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:21.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.854+0000 7f9473fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:21.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.854+0000 7f9473fff640 10 monclient(hunting): my global_id is 28754 2026-02-25T15:08:21.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.854+0000 7f9473fff640 1 -- 10.20.193.90:0/2464761894 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f946c003610 con 0x7f948415b130 2026-02-25T15:08:21.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.855+0000 7f9473fff640 1 -- 10.20.193.90:0/2464761894 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2968695710 0 0) 0x7f948405dd20 con 0x7f94840b29d0 2026-02-25T15:08:21.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.855+0000 7f9473fff640 10 monclient(hunting): _negotiate 2026-02-25T15:08:21.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.855+0000 7f9473fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:21.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.855+0000 7f9473fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:21.859 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.855+0000 7f9473fff640 10 monclient(hunting): my global_id is 30987 2026-02-25T15:08:21.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.855+0000 7f9473fff640 1 -- 10.20.193.90:0/2464761894 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94840b81a0 con 0x7f94840b29d0 2026-02-25T15:08:21.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.855+0000 7f9473fff640 1 -- 10.20.193.90:0/2464761894 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2863589914 0 0) 0x7f94840b7ec0 con 0x7f94840b5d30 2026-02-25T15:08:21.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.855+0000 7f9473fff640 10 monclient(hunting): _negotiate 2026-02-25T15:08:21.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.855+0000 7f9473fff640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:21.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.855+0000 7f9473fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:21.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.855+0000 7f9473fff640 10 monclient(hunting): my global_id is 27616 2026-02-25T15:08:21.860 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.855+0000 7f9473fff640 1 -- 10.20.193.90:0/2464761894 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f948405dd20 con 0x7f94840b5d30 2026-02-25T15:08:21.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f9473fff640 1 -- 10.20.193.90:0/2464761894 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 747656050 0 0) 0x7f946c003610 con 0x7f948415b130 2026-02-25T15:08:21.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f9473fff640 1 -- 10.20.193.90:0/2464761894 >> v1:10.20.193.93:6789/0 conn(0x7f94840b5d30 legacy=0x7f9484059d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:21.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f9473fff640 1 -- 10.20.193.90:0/2464761894 >> v1:10.20.193.90:6789/0 conn(0x7f94840b29d0 legacy=0x7f948405a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:21.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f9473fff640 10 monclient: _finish_hunting 0 2026-02-25T15:08:21.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f9473fff640 1 monclient: found mon.noname-c 2026-02-25T15:08:21.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f9473fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:21.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f9473fff640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.94:6789/0 2026-02-25T15:08:21.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f9473fff640 1 -- 10.20.193.90:0/2464761894 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94840b6760 con 0x7f948415b130 2026-02-25T15:08:21.867 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f9473fff640 10 monclient: _finish_auth 0 2026-02-25T15:08:21.868 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f9473fff640 10 monclient: _check_auth_tickets 2026-02-25T15:08:21.868 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f9473fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:21.868 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f9473fff640 1 -- 10.20.193.90:0/2464761894 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f9474002df0 con 0x7f948415b130 2026-02-25T15:08:21.868 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f9473fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:21.868 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f9473fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:08:21.868 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f9473fff640 10 monclient: dump: 2026-02-25T15:08:21.868 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:21.868 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:21.868 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.868 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.868 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:21.868 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:21.869 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:21.869 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:21.869 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:21.869 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f948b399640 5 monclient: authenticate success, global_id 28754 2026-02-25T15:08:21.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.856+0000 7f948b399640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-25T15:08:21.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.857+0000 7f9473fff640 1 -- 10.20.193.90:0/2464761894 <== mon.2 v1:10.20.193.94:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9474002fc0 con 0x7f948415b130 2026-02-25T15:08:21.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.857+0000 7f9473fff640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:21.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.857+0000 7f9473fff640 1 -- 10.20.193.90:0/2464761894 <== mon.2 v1:10.20.193.94:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f9474003400 con 0x7f948415b130 2026-02-25T15:08:21.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.857+0000 7f9473fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:21.869 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.857+0000 7f9473fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:08:21.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.857+0000 7f9473fff640 10 monclient: dump: 2026-02-25T15:08:21.870 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:21.870 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:21.870 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.870 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.870 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:21.870 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:21.870 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:21.870 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:21.870 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:21.870 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.870 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.857+0000 7f948b399640 10 monclient: get_monmap_and_config success 2026-02-25T15:08:21.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.857+0000 7f948b399640 4 set_mon_vals no callback set 2026-02-25T15:08:21.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.857+0000 7f948b399640 10 monclient: shutdown 2026-02-25T15:08:21.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.857+0000 7f948b399640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:21.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.857+0000 7f948b399640 1 -- 10.20.193.90:0/2464761894 >> v1:10.20.193.94:6789/0 conn(0x7f948415b130 legacy=0x7f94840b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:21.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.857+0000 7f948b399640 1 -- 10.20.193.90:0/2464761894 shutdown_connections 2026-02-25T15:08:21.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.857+0000 7f948b399640 1 -- 10.20.193.90:0/2464761894 >> 10.20.193.90:0/2464761894 conn(0x7f94840b5220 msgr2=0x7f94840b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:21.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.857+0000 7f948b399640 1 -- 10.20.193.90:0/2464761894 shutdown_connections 2026-02-25T15:08:21.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.857+0000 7f948b399640 1 -- 10.20.193.90:0/2464761894 wait complete. 2026-02-25T15:08:21.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.858+0000 7f948b399640 10 monclient: build_initial_monmap 2026-02-25T15:08:21.871 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.858+0000 7f948b399640 10 monclient: monmap: 2026-02-25T15:08:21.871 INFO:teuthology.orchestra.run.trial090.stderr:epoch 0 2026-02-25T15:08:21.872 INFO:teuthology.orchestra.run.trial090.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-25T15:08:21.872 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 0.000000 2026-02-25T15:08:21.872 INFO:teuthology.orchestra.run.trial090.stderr:created 0.000000 2026-02-25T15:08:21.872 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 0 (unknown) 2026-02-25T15:08:21.872 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 1 2026-02-25T15:08:21.872 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.noname-a 2026-02-25T15:08:21.873 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.noname-b 2026-02-25T15:08:21.873 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.noname-c 2026-02-25T15:08:21.873 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.858+0000 7f948b399640 1 librados: starting msgr at 2026-02-25T15:08:21.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.858+0000 7f948b399640 1 librados: starting objecter 2026-02-25T15:08:21.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.858+0000 7f948b399640 20 client.?.objecter Objecter: read policy: balance 2026-02-25T15:08:21.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.858+0000 7f948b399640 1 Processor -- start 2026-02-25T15:08:21.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.858+0000 7f948b399640 1 -- start start 2026-02-25T15:08:21.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.858+0000 7f948b399640 1 librados: setting wanted keys 2026-02-25T15:08:21.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.858+0000 7f948b399640 1 librados: calling monclient init 2026-02-25T15:08:21.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.858+0000 7f948b399640 10 monclient: init 2026-02-25T15:08:21.873 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f948b399640 10 monclient: _reopen_session rank -1 2026-02-25T15:08:21.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f948b399640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-25T15:08:21.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f948b399640 10 monclient(hunting): picked mon.noname-a con 0x7f94840b29d0 addr v1:10.20.193.90:6789/0 2026-02-25T15:08:21.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f948b399640 10 monclient(hunting): picked mon.noname-c con 0x7f94840b5d30 addr v1:10.20.193.94:6789/0 2026-02-25T15:08:21.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f948b399640 10 monclient(hunting): picked mon.noname-b con 0x7f948415b130 addr v1:10.20.193.93:6789/0 2026-02-25T15:08:21.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f948b399640 1 -- --> v1:10.20.193.90:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9484137080 con 0x7f94840b29d0 2026-02-25T15:08:21.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f948b399640 1 -- --> v1:10.20.193.93:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9484133120 con 0x7f948415b130 2026-02-25T15:08:21.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f948b399640 1 -- --> v1:10.20.193.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9484133400 con 0x7f94840b5d30 2026-02-25T15:08:21.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f948b399640 10 monclient(hunting): _renew_subs 2026-02-25T15:08:21.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f948b399640 10 monclient(hunting): authenticate will time out at 3391.459173s 2026-02-25T15:08:21.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f948910e640 1 --1- >> v1:10.20.193.90:6789/0 conn(0x7f94840b29d0 0x7f94841327e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.90:6789/0 says I am v1:10.20.193.90:46114/0 (socket says 10.20.193.90:46114) 2026-02-25T15:08:21.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f948910e640 1 -- 10.20.193.90:0/1537234307 learned_addr learned my addr 10.20.193.90:0/1537234307 (peer_addr_for_me v1:10.20.193.90:0/0) 2026-02-25T15:08:21.874 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f9471ffb640 10 client.?.objecter ms_handle_connect 0x7f94840b29d0 2026-02-25T15:08:21.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f9471ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:21.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f9471ffb640 10 client.?.objecter ms_handle_connect 0x7f94840b5d30 2026-02-25T15:08:21.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f9471ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:21.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f9471ffb640 10 client.?.objecter ms_handle_connect 0x7f948415b130 2026-02-25T15:08:21.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.859+0000 7f9471ffb640 10 client.?.objecter resend_mon_ops 2026-02-25T15:08:21.875 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.2 v1:10.20.193.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1233490797 0 0) 0x7f9484133400 con 0x7f94840b5d30 2026-02-25T15:08:21.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:08:21.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:21.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:21.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 10 monclient(hunting): my global_id is 28757 2026-02-25T15:08:21.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 --> v1:10.20.193.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9454003610 con 0x7f94840b5d30 2026-02-25T15:08:21.876 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.0 v1:10.20.193.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2325355041 0 0) 0x7f9484137080 con 0x7f94840b29d0 2026-02-25T15:08:21.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:08:21.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:21.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:21.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 10 monclient(hunting): my global_id is 30990 2026-02-25T15:08:21.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 --> v1:10.20.193.90:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9484133400 con 0x7f94840b29d0 2026-02-25T15:08:21.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.1 v1:10.20.193.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1861619340 0 0) 0x7f9484133120 con 0x7f948415b130 2026-02-25T15:08:21.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 10 monclient(hunting): _negotiate 2026-02-25T15:08:21.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 10 monclient(hunting): _init_auth method 2 2026-02-25T15:08:21.877 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-25T15:08:21.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 10 monclient(hunting): my global_id is 27619 2026-02-25T15:08:21.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.860+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 --> v1:10.20.193.93:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9484137080 con 0x7f948415b130 2026-02-25T15:08:21.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.862+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.2 v1:10.20.193.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1950144746 0 0) 0x7f9454003610 con 0x7f94840b5d30 2026-02-25T15:08:21.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.862+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 --> v1:10.20.193.94:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9484133120 con 0x7f94840b5d30 2026-02-25T15:08:21.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.862+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.2 v1:10.20.193.94:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f9478003090 con 0x7f94840b5d30 2026-02-25T15:08:21.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.862+0000 7f9471ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:21.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.862+0000 7f9471ffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-25T15:08:21.878 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.862+0000 7f9471ffb640 10 monclient(hunting): dump: 2026-02-25T15:08:21.878 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:21.878 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:21.878 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.878 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.878 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:21.879 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:21.879 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:21.879 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:21.879 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:21.879 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.862+0000 7f9471ffb640 10 monclient(hunting): _finish_auth 0 2026-02-25T15:08:21.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.863+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.1 v1:10.20.193.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1013746866 0 0) 0x7f9484137080 con 0x7f948415b130 2026-02-25T15:08:21.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.863+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 --> v1:10.20.193.93:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9454003610 con 0x7f948415b130 2026-02-25T15:08:21.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.863+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.1 v1:10.20.193.93:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f9480003400 con 0x7f948415b130 2026-02-25T15:08:21.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.863+0000 7f9471ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:21.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.863+0000 7f9471ffb640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-25T15:08:21.879 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.863+0000 7f9471ffb640 10 monclient(hunting): dump: 2026-02-25T15:08:21.880 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:21.880 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:21.880 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.880 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.880 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:21.880 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:21.880 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:21.880 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:21.880 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:21.880 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.880 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.863+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.0 v1:10.20.193.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2676022305 0 0) 0x7f9484133400 con 0x7f94840b29d0 2026-02-25T15:08:21.880 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.863+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 --> v1:10.20.193.90:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9484137080 con 0x7f94840b29d0 2026-02-25T15:08:21.880 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.863+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.0 v1:10.20.193.90:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f9474003310 con 0x7f94840b29d0 2026-02-25T15:08:21.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.863+0000 7f9471ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-25T15:08:21.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.863+0000 7f9471ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-25T15:08:21.881 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.863+0000 7f9471ffb640 10 monclient(hunting): dump: 2026-02-25T15:08:21.881 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:21.881 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:21.881 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.881 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.881 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:21.881 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:21.881 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:21.881 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:21.881 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:21.882 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.2 v1:10.20.193.94:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1404265830 0 0) 0x7f9484133120 con 0x7f94840b5d30 2026-02-25T15:08:21.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 >> v1:10.20.193.93:6789/0 conn(0x7f948415b130 legacy=0x7f9484138610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:21.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 >> v1:10.20.193.90:6789/0 conn(0x7f94840b29d0 legacy=0x7f94841327e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:21.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f9471ffb640 10 monclient: _finish_hunting 0 2026-02-25T15:08:21.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f9471ffb640 1 monclient: found mon.c 2026-02-25T15:08:21.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f9471ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-25T15:08:21.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f9471ffb640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:08:21.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 --> v1:10.20.193.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9484135e90 con 0x7f94840b5d30 2026-02-25T15:08:21.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f9471ffb640 10 monclient: _finish_auth 0 2026-02-25T15:08:21.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f9471ffb640 10 monclient: _check_auth_tickets 2026-02-25T15:08:21.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f9471ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-25T15:08:21.882 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f948b399640 5 monclient: authenticate success, global_id 28757 2026-02-25T15:08:21.883 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f948b399640 10 monclient: _renew_subs 2026-02-25T15:08:21.883 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f948b399640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:08:21.883 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f948b399640 1 -- 10.20.193.90:0/1537234307 --> v1:10.20.193.94:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9484136170 con 0x7f94840b5d30 2026-02-25T15:08:21.883 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f948b399640 10 client.28757.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-25T15:08:21.883 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f948b399640 10 monclient: _renew_subs 2026-02-25T15:08:21.883 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f948b399640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:08:21.883 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f948b399640 1 -- 10.20.193.90:0/1537234307 --> v1:10.20.193.94:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f948411f240 con 0x7f94840b5d30 2026-02-25T15:08:21.883 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f948b399640 1 librados: init done 2026-02-25T15:08:21.883 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f948b399640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:21.884 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f948b399640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:21.884 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f948b399640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:08:21.884 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.864+0000 7f948b399640 1 -- 10.20.193.90:0/1537234307 --> v1:10.20.193.94:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94840b6130 con 0x7f94840b5d30 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.954+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.2 v1:10.20.193.94:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9478003700 con 0x7f94840b5d30 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.954+0000 7f9471ffb640 10 monclient: handle_config config(0 keys) 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.954+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.2 v1:10.20.193.94:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 1088175796 0 0) 0x7f9478003a50 con 0x7f94840b5d30 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.954+0000 7f9471ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.954+0000 7f9471ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.954+0000 7f9471ffb640 10 monclient: dump: 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:epoch 1 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:fsid 63c679b5-d219-4797-8adb-7340f3fd6819 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:last_changed 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:created 2026-02-25T15:00:19.012217+0000 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:min_mon_release 20 (tentacle) 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:election_strategy: 3 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:0: v1:10.20.193.90:6789/0 mon.a 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:1: v1:10.20.193.93:6789/0 mon.b 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:2: v1:10.20.193.94:6789/0 mon.c 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:08:21.956 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.954+0000 7f94737fe640 4 set_mon_vals no callback set 2026-02-25T15:08:21.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.955+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.2 v1:10.20.193.94:6789/0 7 ==== mgrmap(e 5) ==== 103224+0+0 (unknown 1745395530 0 0) 0x7f947801cf30 con 0x7f94840b5d30 2026-02-25T15:08:21.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.956+0000 7f948910e640 10 monclient: get_auth_request con 0x7f9454079fa0 auth_method 0 2026-02-25T15:08:21.957 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.956+0000 7f9471ffb640 10 client.28757.objecter ms_handle_connect 0x7f9454079fa0 2026-02-25T15:08:21.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.956+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.2 v1:10.20.193.94:6789/0 8 ==== osd_map(338..338 src has 1..338) ==== 8959+0+0 (unknown 1125038982 0 0) 0x7f94780948e0 con 0x7f94840b5d30 2026-02-25T15:08:21.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.956+0000 7f9471ffb640 10 client.28757.objecter ms_dispatch2 0x7f9484000b90 osd_map(338..338 src has 1..338) 2026-02-25T15:08:21.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.956+0000 7f9471ffb640 3 client.28757.objecter handle_osd_map got epochs [338,338] > 0 2026-02-25T15:08:21.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.956+0000 7f9471ffb640 3 client.28757.objecter handle_osd_map decoding full epoch 338 2026-02-25T15:08:21.958 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.956+0000 7f9471ffb640 20 client.28757.objecter dump_active .. 0 homeless 2026-02-25T15:08:21.959 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.958+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.2 v1:10.20.193.94:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+232701 (unknown 1092875540 0 2851345850) 0x7f94840b6130 con 0x7f94840b5d30 2026-02-25T15:08:21.959 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.958+0000 7f9471ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-25T15:08:21.959 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:21.958+0000 7f9471ffb640 10 monclient: _finish_command 1 = system:0 2026-02-25T15:08:22.065 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.064+0000 7f948b399640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-25T15:08:22.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.064+0000 7f948b399640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-25T15:08:22.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.064+0000 7f948b399640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.94:6789/0 2026-02-25T15:08:22.066 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.064+0000 7f948b399640 1 -- 10.20.193.90:0/1537234307 --> v1:10.20.193.94:6789/0 -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f9484159590 con 0x7f94840b5d30 2026-02-25T15:08:22.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.174+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.2 v1:10.20.193.94:6789/0 10 ==== config(1 keys) ==== 35+0+0 (unknown 2555425225 0 0) 0x7f947805e240 con 0x7f94840b5d30 2026-02-25T15:08:22.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.174+0000 7f9471ffb640 10 monclient: handle_config config(1 keys) 2026-02-25T15:08:22.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.174+0000 7f94737fe640 4 set_mon_vals no callback set 2026-02-25T15:08:22.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.174+0000 7f9471ffb640 1 -- 10.20.193.90:0/1537234307 <== mon.2 v1:10.20.193.94:6789/0 11 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) ==== 123+0+0 (unknown 691139944 0 0) 0x7f947805db40 con 0x7f94840b5d30 2026-02-25T15:08:22.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.174+0000 7f9471ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-25T15:08:22.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.174+0000 7f9471ffb640 10 monclient: _finish_command 2 = system:0 2026-02-25T15:08:22.176 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.174+0000 7f94737fe640 10 set_mon_vals mon_health_to_clog = false 2026-02-25T15:08:22.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.177+0000 7f948b399640 10 librados: watch_flush enter 2026-02-25T15:08:22.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.177+0000 7f948b399640 10 librados: watch_flush exit 2026-02-25T15:08:22.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.177+0000 7f948b399640 20 client.28757.objecter shutdown clearing up homeless session... 2026-02-25T15:08:22.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.177+0000 7f948b399640 10 client.28757.objecter successfully canceled tick 2026-02-25T15:08:22.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.177+0000 7f948b399640 1 -- 10.20.193.90:0/1537234307 >> v1:10.20.193.189:6800/3647375982 conn(0x7f9454079fa0 legacy=0x7f945409a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:22.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.177+0000 7f948b399640 10 monclient: shutdown 2026-02-25T15:08:22.178 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.177+0000 7f948b399640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-25T15:08:22.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.177+0000 7f948b399640 1 -- 10.20.193.90:0/1537234307 >> v1:10.20.193.94:6789/0 conn(0x7f94840b5d30 legacy=0x7f9484136970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T15:08:22.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.177+0000 7f948b399640 1 -- 10.20.193.90:0/1537234307 shutdown_connections 2026-02-25T15:08:22.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.177+0000 7f948b399640 1 -- 10.20.193.90:0/1537234307 >> 10.20.193.90:0/1537234307 conn(0x7f94840b5220 msgr2=0x7f94840afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T15:08:22.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.177+0000 7f948b399640 1 -- 10.20.193.90:0/1537234307 shutdown_connections 2026-02-25T15:08:22.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.177+0000 7f948b399640 1 -- 10.20.193.90:0/1537234307 wait complete. 2026-02-25T15:08:22.179 INFO:teuthology.orchestra.run.trial090.stderr:2026-02-25T15:08:22.177+0000 7f948b399640 1 librados: shutdown 2026-02-25T15:08:22.188 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-25T15:08:22.188 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-25T15:08:22.189 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-25T15:08:22.189 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:08:22.196 INFO:tasks.ceph.osd.0.trial090.stderr:2026-02-25T15:08:22.190+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 45900) UID: 0 2026-02-25T15:08:22.196 INFO:tasks.ceph.osd.0.trial090.stderr:2026-02-25T15:08:22.191+0000 c763640 -1 osd.0 338 *** Got signal Terminated *** 2026-02-25T15:08:28.290 INFO:tasks.ceph.osd.0:Stopped 2026-02-25T15:08:28.291 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2026-02-25T15:08:28.291 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:08:28.298 INFO:tasks.ceph.osd.4.trial090.stderr:2026-02-25T15:08:28.292+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 45902) UID: 0 2026-02-25T15:08:28.299 INFO:tasks.ceph.osd.4.trial090.stderr:2026-02-25T15:08:28.293+0000 c763640 -1 osd.4 340 *** Got signal Terminated *** 2026-02-25T15:08:28.302 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2026-02-25T15:08:31.465 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:07:44.972 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:31.490 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:07:44.997 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:31.500 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:07:45.007 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:31.513 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:07:45.016 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:31.514 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:07:44.983 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:31.515 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:07:45.017 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:31.515 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:07:44.984 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:31.524 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:07:45.026 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:31.525 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:07:44.994 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:34.392 INFO:tasks.ceph.osd.4:Stopped 2026-02-25T15:08:34.393 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2026-02-25T15:08:34.393 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:08:34.399 INFO:tasks.ceph.osd.8.trial090.stderr:2026-02-25T15:08:34.394+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.8.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 8 (PID: 45905) UID: 0 2026-02-25T15:08:34.399 INFO:tasks.ceph.osd.8.trial090.stderr:2026-02-25T15:08:34.394+0000 c763640 -1 osd.8 342 *** Got signal Terminated *** 2026-02-25T15:08:35.217 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2026-02-25T15:08:35.217 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2026-02-25T15:08:40.494 INFO:tasks.ceph.osd.8:Stopped 2026-02-25T15:08:40.495 DEBUG:tasks.ceph.osd.12:waiting for process to exit 2026-02-25T15:08:40.495 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:08:40.502 INFO:tasks.ceph.osd.12.trial090.stderr:2026-02-25T15:08:40.495+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.12.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 12 (PID: 45908) UID: 0 2026-02-25T15:08:40.502 INFO:tasks.ceph.osd.12.trial090.stderr:2026-02-25T15:08:40.496+0000 c763640 -1 osd.12 344 *** Got signal Terminated *** 2026-02-25T15:08:42.031 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.8 is failed for ~0s 2026-02-25T15:08:42.031 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2026-02-25T15:08:46.596 INFO:tasks.ceph.osd.12:Stopped 2026-02-25T15:08:46.596 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-25T15:08:46.596 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:08:46.603 INFO:tasks.ceph.osd.1.trial093.stderr:2026-02-25T15:08:46.596+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 45224) UID: 0 2026-02-25T15:08:46.603 INFO:tasks.ceph.osd.1.trial093.stderr:2026-02-25T15:08:46.597+0000 c763640 -1 osd.1 348 *** Got signal Terminated *** 2026-02-25T15:08:48.743 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.8 has been restored 2026-02-25T15:08:51.527 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:08:05.034 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:51.529 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:08:05.036 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:51.541 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:08:05.048 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:51.553 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:08:05.055 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:51.554 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:08:05.023 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:51.554 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:08:05.057 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:51.556 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:08:05.025 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:51.566 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:08:05.069 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:51.568 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:08:05.036 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:08:52.697 INFO:tasks.ceph.osd.1:Stopped 2026-02-25T15:08:52.698 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2026-02-25T15:08:52.698 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:08:52.705 INFO:tasks.ceph.osd.5.trial093.stderr:2026-02-25T15:08:52.698+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 45227) UID: 0 2026-02-25T15:08:52.706 INFO:tasks.ceph.osd.5.trial093.stderr:2026-02-25T15:08:52.699+0000 c763640 -1 osd.5 350 *** Got signal Terminated *** 2026-02-25T15:08:55.349 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2026-02-25T15:08:58.799 INFO:tasks.ceph.osd.5:Stopped 2026-02-25T15:08:58.799 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2026-02-25T15:08:58.800 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:08:58.807 INFO:tasks.ceph.osd.9.trial093.stderr:2026-02-25T15:08:58.799+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.9.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 9 (PID: 45229) UID: 0 2026-02-25T15:08:58.807 INFO:tasks.ceph.osd.9.trial093.stderr:2026-02-25T15:08:58.800+0000 c763640 -1 osd.9 352 *** Got signal Terminated *** 2026-02-25T15:09:01.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.9 is failed for ~0s 2026-02-25T15:09:01.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2026-02-25T15:09:04.901 INFO:tasks.ceph.osd.9:Stopped 2026-02-25T15:09:04.901 DEBUG:tasks.ceph.osd.13:waiting for process to exit 2026-02-25T15:09:04.901 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:09:04.909 INFO:tasks.ceph.osd.13.trial093.stderr:2026-02-25T15:09:04.901+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.13.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 13 (PID: 45232) UID: 0 2026-02-25T15:09:04.909 INFO:tasks.ceph.osd.13.trial093.stderr:2026-02-25T15:09:04.902+0000 c763640 -1 osd.13 354 *** Got signal Terminated *** 2026-02-25T15:09:08.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.13 is failed for ~0s 2026-02-25T15:09:08.072 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.9 has been restored 2026-02-25T15:09:11.003 INFO:tasks.ceph.osd.13:Stopped 2026-02-25T15:09:11.003 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-25T15:09:11.004 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:09:11.011 INFO:tasks.ceph.osd.2.trial094.stderr:2026-02-25T15:09:11.004+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 45082) UID: 0 2026-02-25T15:09:11.012 INFO:tasks.ceph.osd.2.trial094.stderr:2026-02-25T15:09:11.006+0000 c763640 -1 osd.2 356 *** Got signal Terminated *** 2026-02-25T15:09:11.614 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:08:25.121 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:11.615 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:08:25.122 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:11.616 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:08:25.085 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:11.617 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:08:25.086 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:11.626 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:08:25.134 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:11.629 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:08:25.098 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:11.640 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:08:25.142 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:11.641 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:08:25.143 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:11.652 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:08:25.154 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:14.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2026-02-25T15:09:14.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.13 has been restored 2026-02-25T15:09:17.105 INFO:tasks.ceph.osd.2:Stopped 2026-02-25T15:09:17.105 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2026-02-25T15:09:17.106 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:09:17.113 INFO:tasks.ceph.osd.6.trial094.stderr:2026-02-25T15:09:17.106+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 45085) UID: 0 2026-02-25T15:09:17.114 INFO:tasks.ceph.osd.6.trial094.stderr:2026-02-25T15:09:17.107+0000 c763640 -1 osd.6 358 *** Got signal Terminated *** 2026-02-25T15:09:20.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2026-02-25T15:09:20.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2026-02-25T15:09:23.208 INFO:tasks.ceph.osd.6:Stopped 2026-02-25T15:09:23.208 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2026-02-25T15:09:23.208 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:09:23.215 INFO:tasks.ceph.osd.10.trial094.stderr:2026-02-25T15:09:23.209+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.10.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 10 (PID: 45086) UID: 0 2026-02-25T15:09:23.215 INFO:tasks.ceph.osd.10.trial094.stderr:2026-02-25T15:09:23.210+0000 c763640 -1 osd.10 360 *** Got signal Terminated *** 2026-02-25T15:09:26.401 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.10 is failed for ~0s 2026-02-25T15:09:26.401 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2026-02-25T15:09:29.310 INFO:tasks.ceph.osd.10:Stopped 2026-02-25T15:09:29.311 DEBUG:tasks.ceph.osd.14:waiting for process to exit 2026-02-25T15:09:29.311 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:09:29.318 INFO:tasks.ceph.osd.14.trial094.stderr:2026-02-25T15:09:29.311+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.14.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 14 (PID: 45090) UID: 0 2026-02-25T15:09:29.318 INFO:tasks.ceph.osd.14.trial094.stderr:2026-02-25T15:09:29.312+0000 c763640 -1 osd.14 362 *** Got signal Terminated *** 2026-02-25T15:09:31.605 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:08:45.112 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:31.607 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:08:45.114 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:31.617 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:08:45.125 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:31.630 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:08:45.132 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:31.631 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:08:45.133 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:31.632 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:08:45.101 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:31.633 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:08:45.102 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:31.642 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:08:45.144 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:31.644 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:08:45.113 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:32.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.10 has been restored 2026-02-25T15:09:35.412 INFO:tasks.ceph.osd.14:Stopped 2026-02-25T15:09:35.412 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-25T15:09:35.412 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:09:35.429 INFO:tasks.ceph.osd.3.trial189.stderr:2026-02-25T15:09:35.425+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 45469) UID: 0 2026-02-25T15:09:35.429 INFO:tasks.ceph.osd.3.trial189.stderr:2026-02-25T15:09:35.426+0000 c763640 -1 osd.3 364 *** Got signal Terminated *** 2026-02-25T15:09:38.217 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2026-02-25T15:09:41.513 INFO:tasks.ceph.osd.3:Stopped 2026-02-25T15:09:41.514 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2026-02-25T15:09:41.514 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:09:41.521 INFO:tasks.ceph.osd.7.trial189.stderr:2026-02-25T15:09:41.516+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 45473) UID: 0 2026-02-25T15:09:41.521 INFO:tasks.ceph.osd.7.trial189.stderr:2026-02-25T15:09:41.517+0000 c763640 -1 osd.7 364 *** Got signal Terminated *** 2026-02-25T15:09:43.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2026-02-25T15:09:43.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2026-02-25T15:09:47.616 INFO:tasks.ceph.osd.7:Stopped 2026-02-25T15:09:47.617 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2026-02-25T15:09:47.617 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:09:47.630 INFO:tasks.ceph.osd.11.trial189.stderr:2026-02-25T15:09:47.625+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.11.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 11 (PID: 45476) UID: 0 2026-02-25T15:09:47.630 INFO:tasks.ceph.osd.11.trial189.stderr:2026-02-25T15:09:47.626+0000 c763640 -1 osd.11 364 *** Got signal Terminated *** 2026-02-25T15:09:49.528 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.11 is failed for ~0s 2026-02-25T15:09:49.528 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2026-02-25T15:09:51.612 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:09:05.119 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:51.636 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:09:05.144 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:51.644 INFO:tasks.ceph.mon.a.trial090.stderr:==00:00:09:05.151 45499== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:51.661 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:09:05.164 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:51.662 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:09:05.165 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:51.664 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:09:05.132 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:51.664 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:09:05.133 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:51.669 INFO:tasks.ceph.mon.b.trial093.stderr:==00:00:09:05.172 44974== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:51.671 INFO:tasks.ceph.mon.c.trial094.stderr:==00:00:09:05.140 44834== Warning: unimplemented fcntl command: 1036 2026-02-25T15:09:53.718 INFO:tasks.ceph.osd.11:Stopped 2026-02-25T15:09:53.719 DEBUG:tasks.ceph.osd.15:waiting for process to exit 2026-02-25T15:09:53.719 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:09:53.726 INFO:tasks.ceph.osd.15.trial189.stderr:2026-02-25T15:09:53.721+0000 c763640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.15.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 15 (PID: 45478) UID: 0 2026-02-25T15:09:53.726 INFO:tasks.ceph.osd.15.trial189.stderr:2026-02-25T15:09:53.722+0000 c763640 -1 osd.15 364 *** Got signal Terminated *** 2026-02-25T15:09:55.034 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 is failed for ~0s 2026-02-25T15:09:55.034 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.11 has been restored 2026-02-25T15:09:59.820 INFO:tasks.ceph.osd.15:Stopped 2026-02-25T15:09:59.821 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-25T15:09:59.821 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2026-02-25T15:09:59.821 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:09:59.854 INFO:tasks.ceph.mgr.y:Stopped 2026-02-25T15:09:59.854 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-25T15:09:59.854 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:09:59.855 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-25T15:09:59.855+0000 7fa9ecda6640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i x (PID: 44397) UID: 0 2026-02-25T15:09:59.855 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-25T15:09:59.855+0000 7fa9ecda6640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2026-02-25T15:09:59.888 INFO:tasks.ceph.mgr.x:Stopped 2026-02-25T15:09:59.889 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-25T15:09:59.889 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-25T15:09:59.889 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:09:59.903 INFO:tasks.ceph.mon.a.trial090.stderr:2026-02-25T15:09:59.896+0000 add5640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 45497) UID: 0 2026-02-25T15:09:59.904 INFO:tasks.ceph.mon.a.trial090.stderr:2026-02-25T15:09:59.896+0000 add5640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2026-02-25T15:10:00.337 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 has been restored 2026-02-25T15:10:05.540 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2026-02-25T15:10:05.991 INFO:tasks.ceph.mon.a:Stopped 2026-02-25T15:10:05.992 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-25T15:10:05.992 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:10:06.000 INFO:tasks.ceph.mon.b.trial093.stderr:2026-02-25T15:10:05.992+0000 add5640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 44972) UID: 0 2026-02-25T15:10:06.001 INFO:tasks.ceph.mon.b.trial093.stderr:2026-02-25T15:10:05.993+0000 add5640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2026-02-25T15:10:10.641 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2026-02-25T15:10:10.642 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2026-02-25T15:10:12.093 INFO:tasks.ceph.mon.b:Stopped 2026-02-25T15:10:12.094 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-25T15:10:12.094 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-25T15:10:12.105 INFO:tasks.ceph.mon.c.trial094.stderr:2026-02-25T15:10:12.095+0000 add5640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 44832) UID: 0 2026-02-25T15:10:12.105 INFO:tasks.ceph.mon.c.trial094.stderr:2026-02-25T15:10:12.095+0000 add5640 -1 mon.c@2(probing) e1 *** Got Signal Terminated *** 2026-02-25T15:10:15.642 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2026-02-25T15:10:15.643 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2026-02-25T15:10:18.195 INFO:tasks.ceph.mon.c:Stopped 2026-02-25T15:10:18.195 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-25T15:10:18.195 DEBUG:teuthology.orchestra.run.trial090:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v 'OSD bench result' | egrep -v OSD_DOWN | head -n 1 2026-02-25T15:10:18.226 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial090.front.sepia.ceph.com 2026-02-25T15:10:18.227 DEBUG:teuthology.orchestra.run.trial090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-25T15:10:18.303 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@trial090.front.sepia.ceph.com 2026-02-25T15:10:18.303 DEBUG:teuthology.orchestra.run.trial090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2026-02-25T15:10:18.378 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@trial090.front.sepia.ceph.com 2026-02-25T15:10:18.378 DEBUG:teuthology.orchestra.run.trial090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2026-02-25T15:10:18.454 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-12 on ubuntu@trial090.front.sepia.ceph.com 2026-02-25T15:10:18.455 DEBUG:teuthology.orchestra.run.trial090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-12 2026-02-25T15:10:18.523 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial093.front.sepia.ceph.com 2026-02-25T15:10:18.524 DEBUG:teuthology.orchestra.run.trial093:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-25T15:10:18.564 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@trial093.front.sepia.ceph.com 2026-02-25T15:10:18.565 DEBUG:teuthology.orchestra.run.trial093:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2026-02-25T15:10:18.639 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@trial093.front.sepia.ceph.com 2026-02-25T15:10:18.640 DEBUG:teuthology.orchestra.run.trial093:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2026-02-25T15:10:18.715 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-13 on ubuntu@trial093.front.sepia.ceph.com 2026-02-25T15:10:18.715 DEBUG:teuthology.orchestra.run.trial093:> sync && sudo umount -f /var/lib/ceph/osd/ceph-13 2026-02-25T15:10:18.787 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial094.front.sepia.ceph.com 2026-02-25T15:10:18.787 DEBUG:teuthology.orchestra.run.trial094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-25T15:10:18.824 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@trial094.front.sepia.ceph.com 2026-02-25T15:10:18.825 DEBUG:teuthology.orchestra.run.trial094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2026-02-25T15:10:18.896 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@trial094.front.sepia.ceph.com 2026-02-25T15:10:18.896 DEBUG:teuthology.orchestra.run.trial094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2026-02-25T15:10:18.971 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-14 on ubuntu@trial094.front.sepia.ceph.com 2026-02-25T15:10:18.971 DEBUG:teuthology.orchestra.run.trial094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-14 2026-02-25T15:10:19.046 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial189.front.sepia.ceph.com 2026-02-25T15:10:19.046 DEBUG:teuthology.orchestra.run.trial189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-25T15:10:19.084 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@trial189.front.sepia.ceph.com 2026-02-25T15:10:19.085 DEBUG:teuthology.orchestra.run.trial189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2026-02-25T15:10:19.153 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@trial189.front.sepia.ceph.com 2026-02-25T15:10:19.153 DEBUG:teuthology.orchestra.run.trial189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2026-02-25T15:10:19.223 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-15 on ubuntu@trial189.front.sepia.ceph.com 2026-02-25T15:10:19.223 DEBUG:teuthology.orchestra.run.trial189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-15 2026-02-25T15:10:19.291 INFO:tasks.ceph:Archiving mon data... 2026-02-25T15:10:19.293 DEBUG:teuthology.misc:Transferring archived files from trial090:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209/data/mon.a.tgz 2026-02-25T15:10:19.293 DEBUG:teuthology.orchestra.run.trial090:> mktemp 2026-02-25T15:10:19.311 INFO:teuthology.orchestra.run.trial090.stdout:/tmp/tmp.uEMQAfqMJb 2026-02-25T15:10:19.311 DEBUG:teuthology.orchestra.run.trial090:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.uEMQAfqMJb 2026-02-25T15:10:19.488 DEBUG:teuthology.orchestra.run.trial090:> sudo chmod 0666 /tmp/tmp.uEMQAfqMJb 2026-02-25T15:10:19.571 DEBUG:teuthology.orchestra.remote:trial090:/tmp/tmp.uEMQAfqMJb is 890KB 2026-02-25T15:10:19.665 DEBUG:teuthology.orchestra.run.trial090:> rm -fr /tmp/tmp.uEMQAfqMJb 2026-02-25T15:10:19.682 DEBUG:teuthology.misc:Transferring archived files from trial093:/var/lib/ceph/mon/ceph-b to /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209/data/mon.b.tgz 2026-02-25T15:10:19.682 DEBUG:teuthology.orchestra.run.trial093:> mktemp 2026-02-25T15:10:19.700 INFO:teuthology.orchestra.run.trial093.stdout:/tmp/tmp.FnX9ekx0KW 2026-02-25T15:10:19.700 DEBUG:teuthology.orchestra.run.trial093:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.FnX9ekx0KW 2026-02-25T15:10:19.879 DEBUG:teuthology.orchestra.run.trial093:> sudo chmod 0666 /tmp/tmp.FnX9ekx0KW 2026-02-25T15:10:19.962 DEBUG:teuthology.orchestra.remote:trial093:/tmp/tmp.FnX9ekx0KW is 890KB 2026-02-25T15:10:20.064 DEBUG:teuthology.orchestra.run.trial093:> rm -fr /tmp/tmp.FnX9ekx0KW 2026-02-25T15:10:20.081 DEBUG:teuthology.misc:Transferring archived files from trial094:/var/lib/ceph/mon/ceph-c to /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209/data/mon.c.tgz 2026-02-25T15:10:20.081 DEBUG:teuthology.orchestra.run.trial094:> mktemp 2026-02-25T15:10:20.099 INFO:teuthology.orchestra.run.trial094.stdout:/tmp/tmp.w1wbHMaopM 2026-02-25T15:10:20.099 DEBUG:teuthology.orchestra.run.trial094:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.w1wbHMaopM 2026-02-25T15:10:20.275 DEBUG:teuthology.orchestra.run.trial094:> sudo chmod 0666 /tmp/tmp.w1wbHMaopM 2026-02-25T15:10:20.357 DEBUG:teuthology.orchestra.remote:trial094:/tmp/tmp.w1wbHMaopM is 890KB 2026-02-25T15:10:20.459 DEBUG:teuthology.orchestra.run.trial094:> rm -fr /tmp/tmp.w1wbHMaopM 2026-02-25T15:10:20.475 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-25T15:10:20.476 DEBUG:teuthology.orchestra.run.trial090:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-25T15:10:20.479 DEBUG:teuthology.orchestra.run.trial093:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-25T15:10:20.482 DEBUG:teuthology.orchestra.run.trial094:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-25T15:10:20.518 DEBUG:teuthology.orchestra.run.trial189:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-25T15:10:20.643 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2026-02-25T15:10:20.802 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-25T15:10:21.033 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-25T15:10:21.293 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-25T15:10:21.505 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-25T15:10:21.505 INFO:tasks.ceph:Archiving crash dumps... 2026-02-25T15:10:21.508 DEBUG:teuthology.misc:Transferring archived files from trial090:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209/remote/trial090/crash 2026-02-25T15:10:21.510 DEBUG:teuthology.orchestra.run.trial090:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-25T15:10:21.537 DEBUG:teuthology.misc:Transferring archived files from trial093:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209/remote/trial093/crash 2026-02-25T15:10:21.539 DEBUG:teuthology.orchestra.run.trial093:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-25T15:10:21.565 DEBUG:teuthology.misc:Transferring archived files from trial094:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209/remote/trial094/crash 2026-02-25T15:10:21.567 DEBUG:teuthology.orchestra.run.trial094:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-25T15:10:21.593 DEBUG:teuthology.misc:Transferring archived files from trial189:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209/remote/trial189/crash 2026-02-25T15:10:21.594 DEBUG:teuthology.orchestra.run.trial189:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-25T15:10:21.621 INFO:tasks.ceph:Compressing logs... 2026-02-25T15:10:21.622 DEBUG:teuthology.orchestra.run.trial090:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T15:10:21.625 DEBUG:teuthology.orchestra.run.trial093:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T15:10:21.627 DEBUG:teuthology.orchestra.run.trial094:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T15:10:21.637 DEBUG:teuthology.orchestra.run.trial189:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T15:10:21.646 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54084.log 2026-02-25T15:10:21.661 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2026-02-25T15:10:21.661 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.5.log 2026-02-25T15:10:21.661 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.13.log 2026-02-25T15:10:21.661 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/valgrind/osd.5.log: gzip 66.0% -- replaced with /var/log/ceph/valgrind/osd.5.log.gz -5 2026-02-25T15:10:21.661 INFO:teuthology.orchestra.run.trial093.stderr: --verbose -- /var/log/ceph/valgrind/osd.1.log 2026-02-25T15:10:21.661 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.9.log 2026-02-25T15:10:21.661 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/valgrind/osd.13.log: 66.0% -- replaced with /var/log/ceph/valgrind/osd.13.log.gz 2026-02-25T15:10:21.661 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/valgrind/osd.1.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.b.log 2026-02-25T15:10:21.661 INFO:teuthology.orchestra.run.trial093.stderr: 66.0% -- replaced with /var/log/ceph/valgrind/osd.1.log.gz 2026-02-25T15:10:21.662 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/valgrind/osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-25T15:10:21.662 INFO:teuthology.orchestra.run.trial093.stderr: 66.0% -- replaced with /var/log/ceph/valgrind/osd.9.log.gz 2026-02-25T15:10:21.662 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/valgrind/mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2026-02-25T15:10:21.662 INFO:teuthology.orchestra.run.trial093.stderr: 65.6% -- replaced with /var/log/ceph/valgrind/mon.b.log.gz 2026-02-25T15:10:21.662 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.13.log 2026-02-25T15:10:21.662 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-mon.b.log: 91.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-25T15:10:21.662 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-25T15:10:21.662 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-osd.13.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-25T15:10:21.662 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.9.log 2026-02-25T15:10:21.662 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-osd.9.log: 88.0% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-25T15:10:21.670 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-25T15:10:21.670 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2026-02-25T15:10:21.670 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.2.log 2026-02-25T15:10:21.670 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.6.log 2026-02-25T15:10:21.670 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/valgrind/osd.2.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.14.log 2026-02-25T15:10:21.670 INFO:teuthology.orchestra.run.trial094.stderr: 65.7% -- replaced with /var/log/ceph/valgrind/osd.2.log.gz 2026-02-25T15:10:21.670 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/valgrind/osd.6.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.10.log 2026-02-25T15:10:21.670 INFO:teuthology.orchestra.run.trial094.stderr: 66.0% -- replaced with /var/log/ceph/valgrind/osd.6.log.gz 2026-02-25T15:10:21.671 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/valgrind/osd.14.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.c.log 2026-02-25T15:10:21.671 INFO:teuthology.orchestra.run.trial094.stderr: 66.0% -- replaced with /var/log/ceph/valgrind/osd.14.log.gz 2026-02-25T15:10:21.671 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/valgrind/osd.10.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.10.log 2026-02-25T15:10:21.671 INFO:teuthology.orchestra.run.trial094.stderr: 66.0% -- replaced with /var/log/ceph/valgrind/osd.10.log.gz 2026-02-25T15:10:21.671 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/valgrind/mon.c.log: 65.6%gzip -- replaced with /var/log/ceph/valgrind/mon.c.log.gz 2026-02-25T15:10:21.671 INFO:teuthology.orchestra.run.trial094.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.14.log 2026-02-25T15:10:21.671 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-25T15:10:21.671 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-osd.14.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2026-02-25T15:10:21.671 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-25T15:10:21.671 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph.log: 88.0% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-25T15:10:21.671 INFO:teuthology.orchestra.run.trial094.stderr: 91.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-25T15:10:21.691 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54182.log 2026-02-25T15:10:21.691 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.54084.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-25T15:10:21.691 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.54182.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54084.log.gz 2026-02-25T15:10:21.691 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49482.log 2026-02-25T15:10:21.691 INFO:teuthology.orchestra.run.trial090.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54182.log.gz 2026-02-25T15:10:21.691 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-25T15:10:21.691 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.49482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52422.log 2026-02-25T15:10:21.691 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-osd.0.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49482.log.gz 2026-02-25T15:10:21.691 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2026-02-25T15:10:21.691 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.52422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50289.log 2026-02-25T15:10:21.691 INFO:teuthology.orchestra.run.trial090.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.52422.log.gz 2026-02-25T15:10:21.691 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51001.log 2026-02-25T15:10:21.691 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54523.log 2026-02-25T15:10:21.691 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50289.log: /var/log/ceph/ceph-client.admin.51001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51637.log 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50289.log.gz 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.51001.log.gz 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.54523.log: -- /var/log/ceph/ceph-client.admin.50285.log 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52132.log 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54523.log.gz 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50285.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.51637.log.gz 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46009.log 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50285.log.gz 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.52132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51958.log 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.46009.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.52132.log.gz 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54280.log 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46009.log.gz 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54378.log 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.54280.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51958.log.gz 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49755.log 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.54378.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54280.log.gz 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53349.log 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.49755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49641.log 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54378.log.gz 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53349.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49755.log.gz 2026-02-25T15:10:21.692 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51782.log 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.49641.log: gzip -5 87.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.53349.log.gz /var/log/ceph/ceph-client.admin.52330.log 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53937.log 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49641.log.gz 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.52330.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.51782.log.gz 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53398.log 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.52330.log.gz 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50843.log 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51215.log 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53937.log.gz 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53398.log.gz 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50278.log 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.50843.log.gz 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50306.log 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50278.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.51215.log.gz 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54035.log 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50306.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50278.log.gz 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50946.log 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50306.log.gz 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.54035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51638.log 2026-02-25T15:10:21.693 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50946.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54035.log.gz 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53153.log 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51638.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.50946.log.gz 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53741.log 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53153.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.51638.log.gz 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51470.log 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53153.log.gz 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52301.log 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53496.log 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53741.log.gz 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.52301.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51470.log.gz 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53986.log 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.52301.log.gz 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr:gzip/var/log/ceph/ceph-client.admin.53496.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.53006.log 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52908.log 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53496.log.gz 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.53986.log.gz 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53545.log 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr: 87.1%/var/log/ceph/ceph-client.admin.52908.log: -- replaced with /var/log/ceph/ceph-client.admin.53006.log.gz 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.0.log 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53545.log: gzip -5 87.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.52908.log.gz /var/log/ceph/valgrind/osd.12.log 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53545.log.gz 2026-02-25T15:10:21.694 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/valgrind/osd.0.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.8.log 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr: 65.7% -- replaced with /var/log/ceph/valgrind/osd.0.log.gz 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/valgrind/osd.12.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.a.log 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr: 65.5% -- replaced with /var/log/ceph/valgrind/osd.12.log.gz 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/valgrind/osd.8.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.4.log 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr: 66.0% -- replaced with /var/log/ceph/valgrind/osd.8.log.gz 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/valgrind/mon.a.log: gzip 65.6% -5 --verbose -- /var/log/ceph/ceph-client.admin.50270.log 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr: -- replaced with /var/log/ceph/valgrind/mon.a.log.gz 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/valgrind/osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52499.log 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr: 65.6% -- replaced with /var/log/ceph/valgrind/osd.4.log.gz 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47787.log 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50270.log.gz 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.52499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52419.log 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.47787.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52499.log.gz 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53300.log 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.52419.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47787.log.gz 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54476.log 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.52419.log.gz 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51854.log 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.54476.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53300.log.gz 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51010.log 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51854.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54476.log.gz 2026-02-25T15:10:21.695 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53251.log 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51010.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51854.log.gz 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50283.log 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.51010.log.gz 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49853.log 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51217.log 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53251.log.gz 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50292.log 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.49853.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50283.log.gz 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50266.log 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49853.log.gz 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.51217.log.gz 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49618.log 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50266.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50292.log.gz 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49706.log 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.49618.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50266.log.gz 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.12.log 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.49618.log.gz 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.49706.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.39201.log 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-osd.12.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54329.log 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49706.log.gz 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph.tmp-client.admin.39201.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.39201.log.gz 2026-02-25T15:10:21.696 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.54329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51564.log 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54329.log.gz 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50930.log 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45940.log 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50304.log 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50930.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.51564.log.gz 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.45940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53692.log 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.50930.log.gz 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.45940.log.gz 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50293.log 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50304.log.gz 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51009.log 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53692.log.gz 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50992.log 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51506.log 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50293.log.gz 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.51009.log.gz 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log/var/log/ceph/ceph-client.admin.50992.log: 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52170.log 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.50992.log.gz 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.51506.log.gz 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53104.log 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.52170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53643.log 2026-02-25T15:10:21.697 INFO:teuthology.orchestra.run.trial090.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.52170.log.gz 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54231.log 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53104.log: /var/log/ceph/ceph-client.admin.53643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50899.log 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53104.log.gz 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.54231.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53643.log.gz 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49548.log 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51966.log 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54231.log.gz 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.49548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52254.log 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.50899.log.gz 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49548.log.gz 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52597.log 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.52254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50268.log 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51966.log.gz 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52254.log.gz 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:gzip/var/log/ceph/ceph-client.admin.52597.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.53202.log 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53447.log 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50268.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52597.log.gz 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49804.log 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50268.log.gz 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53447.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53202.log.gz 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr: 91.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53790.log 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53447.log.gz 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:gzip/var/log/ceph/ceph-client.admin.49804.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50844.log 2026-02-25T15:10:21.698 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53839.log 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49804.log.gz 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50844.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53790.log.gz 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52663.log 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.50844.log.gz 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50305.log 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.52663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50938.log 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53839.log.gz 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52548.log 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52663.log.gz 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53055.log 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50305.log.gz 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.52548.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.50938.log.gz 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51002.log 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52548.log.gz 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51218.log 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52957.log 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53055.log.gz 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.51002.log.gz 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54427.log 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.52957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50307.log 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.51218.log.gz 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52957.log.gz 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.54427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45550.log 2026-02-25T15:10:21.699 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50839.log 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.45550.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54427.log.gz 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50307.log.gz 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52859.log 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50839.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.45550.log.gz 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.52859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50294.log 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50839.log.gz 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50295.log 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52859.log.gz 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45963.log 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50295.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50294.log.gz 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50295.log.gz 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.45963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51965.log 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45963.log.gz 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52761.log 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52712.log 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.52761.log: 88.0% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45657.log 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.51965.log.gz 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.52712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51956.log 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52761.log.gz 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52712.log.gz 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.45657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54133.log 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53888.log 2026-02-25T15:10:21.700 INFO:teuthology.orchestra.run.trial090.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.45657.log.gz 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.54133.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.51956.log.gz 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52810.log 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53888.log: gzip -5 --verbose 87.7% -- /var/log/ceph/ceph-client.admin.53594.log -- replaced with /var/log/ceph/ceph-client.admin.54133.log.gz 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.52810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52045.log 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53888.log.gz 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.53594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51686.log 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52810.log.gz 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51778.log 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53594.log.gz 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.52045.log: /var/log/ceph/ceph-client.admin.51686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50272.log 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.52045.log.gz 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.51778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50996.log 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.51686.log.gz 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50272.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.51778.log.gz 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr:/var/log/ceph/ceph-client.admin.50996.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50272.log.gz 2026-02-25T15:10:21.701 INFO:teuthology.orchestra.run.trial090.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.50996.log.gz 2026-02-25T15:10:21.710 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50108.log 2026-02-25T15:10:21.710 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49274.log 2026-02-25T15:10:21.710 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.50108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49451.log 2026-02-25T15:10:21.710 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50100.log 2026-02-25T15:10:21.710 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50207.log 2026-02-25T15:10:21.710 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.50100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52211.log 2026-02-25T15:10:21.710 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.50207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44544.log 2026-02-25T15:10:21.710 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.52211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45009.log 2026-02-25T15:10:21.710 INFO:teuthology.orchestra.run.trial189.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50100.log.gz 2026-02-25T15:10:21.710 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44544.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-25T15:10:21.710 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.45009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49765.log 2026-02-25T15:10:21.710 INFO:teuthology.orchestra.run.trial189.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44544.log.gz 2026-02-25T15:10:21.710 INFO:teuthology.orchestra.run.trial189.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45009.log.gz 2026-02-25T15:10:21.710 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49389.log 2026-02-25T15:10:21.710 INFO:teuthology.orchestra.run.trial189.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50207.log.gz 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45197.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49765.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.52211.log.gz 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50260.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.45197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49221.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49451.log.gz 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.50260.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45197.log.gz 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50235.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49231.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr: 90.3%/var/log/ceph/ceph-client.admin.50235.log: -- replaced with /var/log/ceph/ceph-client.admin.49765.log.gz 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52213.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44591.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.52213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52212.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49999.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44591.log.gz 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.52212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51195.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52210.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.51195.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50260.log.gz 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.52213.log.gz 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45150.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.52210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49583.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.45150.log: gzip -5 --verbose -- 90.3% /var/log/ceph/ceph-client.admin.49699.log -- replaced with /var/log/ceph/ceph-client.admin.52212.log.gz 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45150.log.gz 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56231.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.51195.log.gz 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49334.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.52210.log.gz 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.56231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51196.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49334.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.49274.log.gz 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49603.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.51196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51325.log 2026-02-25T15:10:21.711 INFO:teuthology.orchestra.run.trial189.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.50235.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49999.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49314.log 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.51325.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.56231.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49583.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49314.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.51325.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51437.log 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.51196.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50275.log 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.49334.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.51437.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.7.log 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.50275.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.11.log 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/valgrind/osd.7.log: 66.0% -- replaced with /var/log/ceph/valgrind/osd.7.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.15.log 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.3.log 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/valgrind/osd.11.log: /var/log/ceph/valgrind/osd.15.log: 65.9% -- replaced with /var/log/ceph/valgrind/osd.11.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44962.log 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr: 66.0% -- replaced with /var/log/ceph/valgrind/osd.15.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.51437.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49603.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/valgrind/osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50020.log 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr: 66.0% -- replaced with /var/log/ceph/valgrind/osd.3.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.49699.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45259.log 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr: 87.7%/var/log/ceph/ceph-client.admin.50020.log: -- replaced with /var/log/ceph/ceph-client.admin.44962.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.11.log 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.45259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50212.log 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50020.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.45259.log.gz 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49261.log 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.50212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51326.log 2026-02-25T15:10:21.712 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51950.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.51326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49836.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.51950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44649.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50042.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.50108.log.gz 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44649.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.49836.log.gz 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52209.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.51326.log.gz 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.50042.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44649.log.gz 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44743.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.52209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50144.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44743.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.51950.log.gz 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49945.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.44743.log.gz 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.50144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51197.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49945.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49261.log.gz 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49660.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.52209.log.gz 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.51197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49969.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.50042.log.gz 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45103.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49722.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.49945.log.gz 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.45103.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49969.log.gz 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56233.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45103.log.gz 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49501.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49722.log: /var/log/ceph/ceph-client.admin.56233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49980.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.51197.log.gz 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44915.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49785.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.50212.log.gz 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50054.log 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.56233.log.gz 2026-02-25T15:10:21.713 INFO:teuthology.orchestra.run.trial189.stderr: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.44915.log.gz -5 2026-02-25T15:10:21.714 INFO:teuthology.orchestra.run.trial189.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49476.log 2026-02-25T15:10:21.714 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49785.log: /var/log/ceph/ceph-client.admin.50054.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49221.log.gz 2026-02-25T15:10:21.714 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45056.log 2026-02-25T15:10:21.714 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45306.log 2026-02-25T15:10:21.714 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.45056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49947.log 2026-02-25T15:10:21.714 INFO:teuthology.orchestra.run.trial189.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.49980.log.gz 2026-02-25T15:10:21.714 INFO:teuthology.orchestra.run.trial189.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45056.log.gz 2026-02-25T15:10:21.714 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.45306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50048.log 2026-02-25T15:10:21.716 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49947.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45306.log.gz 2026-02-25T15:10:21.716 INFO:teuthology.orchestra.run.trial189.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49660.log.gz 2026-02-25T15:10:21.716 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49541.log 2026-02-25T15:10:21.716 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51438.log 2026-02-25T15:10:21.716 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.50048.log: /var/log/ceph/ceph-client.admin.49541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51951.log 2026-02-25T15:10:21.716 INFO:teuthology.orchestra.run.trial189.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50048.log.gz 2026-02-25T15:10:21.716 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.51438.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49722.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49714.log 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.51951.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.49947.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.51438.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44696.log 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51949.log 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.49476.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56232.log 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.51949.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44696.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49804.log 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49541.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.49501.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.56232.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.50054.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51324.log 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.51951.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49986.log 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.51324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49295.log 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50176.log 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49295.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.49986.log.gz -5 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr: --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.50176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49441.log 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.51949.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.56232.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45353.log 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44843.log 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.51324.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.45353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49363.log 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45353.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44843.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.15.log 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.49295.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr: 91.0% 85.9% -- replaced with /var/log/ceph/ceph-client.admin.49714.log.gz -- replaced with /var/log/ceph/ceph-client.admin.44843.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.49363.log: /var/log/ceph/ceph-osd.15.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.49804.log.gz 2026-02-25T15:10:21.717 INFO:teuthology.orchestra.run.trial189.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.50144.log.gz 2026-02-25T15:10:21.718 INFO:teuthology.orchestra.run.trial189.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.49363.log.gz 2026-02-25T15:10:21.718 INFO:teuthology.orchestra.run.trial189.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.50176.log.gz 2026-02-25T15:10:21.720 INFO:teuthology.orchestra.run.trial189.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.50275.log.gz 2026-02-25T15:10:21.727 INFO:teuthology.orchestra.run.trial189.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.49231.log.gz 2026-02-25T15:10:21.729 INFO:teuthology.orchestra.run.trial189.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.49314.log.gz 2026-02-25T15:10:21.732 INFO:teuthology.orchestra.run.trial189.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.49785.log.gz 2026-02-25T15:10:21.757 INFO:teuthology.orchestra.run.trial189.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.49441.log.gz 2026-02-25T15:10:21.781 INFO:teuthology.orchestra.run.trial189.stderr: 89.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-25T15:10:22.159 INFO:teuthology.orchestra.run.trial189.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49389.log.gz 2026-02-25T15:10:22.249 INFO:teuthology.orchestra.run.trial093.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2026-02-25T15:10:22.308 INFO:teuthology.orchestra.run.trial094.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2026-02-25T15:10:22.347 INFO:teuthology.orchestra.run.trial094.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-25T15:10:22.359 INFO:teuthology.orchestra.run.trial093.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-25T15:10:22.374 INFO:teuthology.orchestra.run.trial090.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-25T15:10:22.414 INFO:teuthology.orchestra.run.trial093.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2026-02-25T15:10:22.421 INFO:teuthology.orchestra.run.trial189.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-25T15:10:22.423 INFO:teuthology.orchestra.run.trial090.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2026-02-25T15:10:22.442 INFO:teuthology.orchestra.run.trial094.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2026-02-25T15:10:22.445 INFO:teuthology.orchestra.run.trial090.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2026-02-25T15:10:22.448 INFO:teuthology.orchestra.run.trial093.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2026-02-25T15:10:22.456 INFO:teuthology.orchestra.run.trial189.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.15.log.gz 2026-02-25T15:10:22.467 INFO:teuthology.orchestra.run.trial093.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.13.log.gz 2026-02-25T15:10:22.468 INFO:teuthology.orchestra.run.trial094.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.14.log.gz 2026-02-25T15:10:22.469 INFO:teuthology.orchestra.run.trial093.stderr: 2026-02-25T15:10:22.470 INFO:teuthology.orchestra.run.trial093.stderr:real 0m0.829s 2026-02-25T15:10:22.470 INFO:teuthology.orchestra.run.trial093.stderr:user 0m3.512s 2026-02-25T15:10:22.470 INFO:teuthology.orchestra.run.trial093.stderr:sys 0m0.187s 2026-02-25T15:10:22.475 INFO:teuthology.orchestra.run.trial094.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.10.log.gz 2026-02-25T15:10:22.477 INFO:teuthology.orchestra.run.trial094.stderr: 2026-02-25T15:10:22.477 INFO:teuthology.orchestra.run.trial094.stderr:real 0m0.827s 2026-02-25T15:10:22.477 INFO:teuthology.orchestra.run.trial094.stderr:user 0m3.581s 2026-02-25T15:10:22.477 INFO:teuthology.orchestra.run.trial094.stderr:sys 0m0.184s 2026-02-25T15:10:22.487 INFO:teuthology.orchestra.run.trial189.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2026-02-25T15:10:22.530 INFO:teuthology.orchestra.run.trial090.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.12.log.gz 2026-02-25T15:10:22.684 INFO:teuthology.orchestra.run.trial189.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.11.log.gz 2026-02-25T15:10:22.686 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-25T15:10:22.687 INFO:teuthology.orchestra.run.trial189.stderr:real 0m1.004s 2026-02-25T15:10:22.687 INFO:teuthology.orchestra.run.trial189.stderr:user 0m3.856s 2026-02-25T15:10:22.687 INFO:teuthology.orchestra.run.trial189.stderr:sys 0m0.275s 2026-02-25T15:10:22.703 INFO:teuthology.orchestra.run.trial090.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-25T15:10:22.705 INFO:teuthology.orchestra.run.trial090.stderr: 2026-02-25T15:10:22.705 INFO:teuthology.orchestra.run.trial090.stderr:real 0m1.067s 2026-02-25T15:10:22.705 INFO:teuthology.orchestra.run.trial090.stderr:user 0m4.039s 2026-02-25T15:10:22.705 INFO:teuthology.orchestra.run.trial090.stderr:sys 0m0.259s 2026-02-25T15:10:22.706 INFO:tasks.ceph:Archiving logs... 2026-02-25T15:10:22.707 DEBUG:teuthology.misc:Transferring archived files from trial090:/var/log/ceph to /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209/remote/trial090/log 2026-02-25T15:10:22.708 DEBUG:teuthology.orchestra.run.trial090:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-25T15:10:23.259 DEBUG:teuthology.misc:Transferring archived files from trial093:/var/log/ceph to /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209/remote/trial093/log 2026-02-25T15:10:23.259 DEBUG:teuthology.orchestra.run.trial093:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-25T15:10:23.594 DEBUG:teuthology.misc:Transferring archived files from trial094:/var/log/ceph to /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209/remote/trial094/log 2026-02-25T15:10:23.595 DEBUG:teuthology.orchestra.run.trial094:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-25T15:10:23.937 DEBUG:teuthology.misc:Transferring archived files from trial189:/var/log/ceph to /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209/remote/trial189/log 2026-02-25T15:10:23.938 DEBUG:teuthology.orchestra.run.trial189:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-25T15:10:24.364 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-25T15:10:24.380 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-25T15:10:24.380 DEBUG:teuthology.orchestra.run.trial090:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-25T15:10:24.382 DEBUG:teuthology.orchestra.run.trial093:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-25T15:10:24.384 DEBUG:teuthology.orchestra.run.trial094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-25T15:10:24.386 DEBUG:teuthology.orchestra.run.trial189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-25T15:10:24.643 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-25T15:10:24.653 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-25T15:10:24.653 DEBUG:teuthology.orchestra.run.trial090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T15:10:24.655 DEBUG:teuthology.orchestra.run.trial093:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T15:10:24.657 DEBUG:teuthology.orchestra.run.trial094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T15:10:24.659 DEBUG:teuthology.orchestra.run.trial189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T15:10:24.670 INFO:teuthology.orchestra.run.trial090.stderr:bash: line 1: ntpq: command not found 2026-02-25T15:10:24.672 INFO:teuthology.orchestra.run.trial093.stderr:bash: line 1: ntpq: command not found 2026-02-25T15:10:24.673 INFO:teuthology.orchestra.run.trial094.stderr:bash: line 1: ntpq: command not found 2026-02-25T15:10:24.675 INFO:teuthology.orchestra.run.trial189.stderr:bash: line 1: ntpq: command not found 2026-02-25T15:10:24.707 INFO:teuthology.orchestra.run.trial094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T15:10:24.707 INFO:teuthology.orchestra.run.trial094.stdout:=============================================================================== 2026-02-25T15:10:24.707 INFO:teuthology.orchestra.run.trial094.stdout:^+ ip74-208-14-149.pbiaas.c> 2 7 377 87 +212us[ +169us] +/- 25ms 2026-02-25T15:10:24.707 INFO:teuthology.orchestra.run.trial094.stdout:^+ 44.190.5.123 2 8 377 28 +2011us[+1964us] +/- 34ms 2026-02-25T15:10:24.707 INFO:teuthology.orchestra.run.trial094.stdout:^* s2-b.time.mci1.us.rozint> 2 7 377 24 +1105us[+1057us] +/- 23ms 2026-02-25T15:10:24.707 INFO:teuthology.orchestra.run.trial094.stdout:^+ 38.134.40.65 2 7 377 480 -2522us[-2607us] +/- 24ms 2026-02-25T15:10:24.708 INFO:teuthology.orchestra.run.trial189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T15:10:24.708 INFO:teuthology.orchestra.run.trial189.stdout:=============================================================================== 2026-02-25T15:10:24.708 INFO:teuthology.orchestra.run.trial189.stdout:^* rn-02.koehn.com 2 8 377 219 -622us[ -673us] +/- 12ms 2026-02-25T15:10:24.708 INFO:teuthology.orchestra.run.trial189.stdout:^+ 44.190.5.123 2 6 377 221 +2002us[+1950us] +/- 33ms 2026-02-25T15:10:24.708 INFO:teuthology.orchestra.run.trial189.stdout:^+ s2-b.time.mci1.us.rozint> 2 6 377 25 +1723us[+1723us] +/- 23ms 2026-02-25T15:10:24.708 INFO:teuthology.orchestra.run.trial189.stdout:^+ 38.134.40.65 2 7 377 94 -2169us[-2169us] +/- 24ms 2026-02-25T15:10:24.708 INFO:teuthology.orchestra.run.trial090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T15:10:24.708 INFO:teuthology.orchestra.run.trial090.stdout:=============================================================================== 2026-02-25T15:10:24.708 INFO:teuthology.orchestra.run.trial090.stdout:^- 172-233-189-68.ip.linode> 2 9 377 355 -2968us[-3227us] +/- 85ms 2026-02-25T15:10:24.708 INFO:teuthology.orchestra.run.trial090.stdout:^+ 44.190.5.123 2 6 377 26 +1594us[+1594us] +/- 34ms 2026-02-25T15:10:24.708 INFO:teuthology.orchestra.run.trial090.stdout:^* s2-b.time.mci1.us.rozint> 2 6 377 26 +1407us[+1399us] +/- 23ms 2026-02-25T15:10:24.708 INFO:teuthology.orchestra.run.trial090.stdout:^+ 38.134.40.65 2 6 377 21 -2497us[-2497us] +/- 24ms 2026-02-25T15:10:24.709 INFO:teuthology.orchestra.run.trial093.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T15:10:24.709 INFO:teuthology.orchestra.run.trial093.stdout:=============================================================================== 2026-02-25T15:10:24.709 INFO:teuthology.orchestra.run.trial093.stdout:^- 172-233-189-68.ip.linode> 2 9 377 348 -4472us[-4527us] +/- 85ms 2026-02-25T15:10:24.709 INFO:teuthology.orchestra.run.trial093.stdout:^+ 44.190.5.123 2 9 377 101 +13us[ +13us] +/- 34ms 2026-02-25T15:10:24.709 INFO:teuthology.orchestra.run.trial093.stdout:^* s2-b.time.mci1.us.rozint> 2 8 377 217 -172us[ -232us] +/- 23ms 2026-02-25T15:10:24.709 INFO:teuthology.orchestra.run.trial093.stdout:^- 38.134.40.65 2 7 377 89 -4117us[-4117us] +/- 24ms 2026-02-25T15:10:24.709 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-25T15:10:24.720 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-25T15:10:24.721 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-25T15:10:24.733 DEBUG:teuthology.orchestra.run.trial090:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-25T15:10:24.804 DEBUG:teuthology.orchestra.run.trial093:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-25T15:10:24.859 DEBUG:teuthology.orchestra.run.trial094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-25T15:10:24.915 DEBUG:teuthology.orchestra.run.trial189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-25T15:10:24.971 DEBUG:teuthology.orchestra.run.trial090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T15:10:25.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T15:10:25.001 DEBUG:teuthology.orchestra.run.trial093:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T15:10:25.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T15:10:25.028 DEBUG:teuthology.orchestra.run.trial094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T15:10:25.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T15:10:25.057 DEBUG:teuthology.orchestra.run.trial189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T15:10:25.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T15:10:25.085 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-25T15:10:25.105 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-25T15:10:25.119 INFO:teuthology.task.internal:Duration was 3176.660555 seconds 2026-02-25T15:10:25.120 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-25T15:10:25.134 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-25T15:10:25.135 DEBUG:teuthology.orchestra.run.trial090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-25T15:10:25.138 DEBUG:teuthology.orchestra.run.trial093:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-25T15:10:25.141 DEBUG:teuthology.orchestra.run.trial094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-25T15:10:25.144 DEBUG:teuthology.orchestra.run.trial189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-25T15:10:25.173 INFO:teuthology.orchestra.run.trial090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T15:10:25.176 INFO:teuthology.orchestra.run.trial093.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T15:10:25.178 INFO:teuthology.orchestra.run.trial094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T15:10:25.181 INFO:teuthology.orchestra.run.trial189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T15:10:25.639 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-25T15:10:25.639 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial090.front.sepia.ceph.com 2026-02-25T15:10:25.639 DEBUG:teuthology.orchestra.run.trial090:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-25T15:10:25.660 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial093.front.sepia.ceph.com 2026-02-25T15:10:25.660 DEBUG:teuthology.orchestra.run.trial093:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-25T15:10:25.698 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial094.front.sepia.ceph.com 2026-02-25T15:10:25.699 DEBUG:teuthology.orchestra.run.trial094:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-25T15:10:25.727 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial189.front.sepia.ceph.com 2026-02-25T15:10:25.743 DEBUG:teuthology.orchestra.run.trial189:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-25T15:10:25.764 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-25T15:10:25.764 DEBUG:teuthology.orchestra.run.trial090:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T15:10:25.767 DEBUG:teuthology.orchestra.run.trial093:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T15:10:25.772 DEBUG:teuthology.orchestra.run.trial094:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T15:10:25.778 DEBUG:teuthology.orchestra.run.trial189:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T15:10:25.869 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-25T15:10:25.870 DEBUG:teuthology.orchestra.run.trial090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T15:10:25.872 DEBUG:teuthology.orchestra.run.trial093:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T15:10:25.875 DEBUG:teuthology.orchestra.run.trial094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T15:10:25.877 DEBUG:teuthology.orchestra.run.trial189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T15:10:25.892 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T15:10:25.894 INFO:teuthology.orchestra.run.trial090.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T15:10:25.894 INFO:teuthology.orchestra.run.trial090.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T15:10:25.894 INFO:teuthology.orchestra.run.trial090.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-25T15:10:25.894 INFO:teuthology.orchestra.run.trial090.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-25T15:10:25.895 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T15:10:25.897 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T15:10:25.897 INFO:teuthology.orchestra.run.trial093.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-25T15:10:25.897 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T15:10:25.897 INFO:teuthology.orchestra.run.trial093.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-25T15:10:25.898 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T15:10:25.898 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T15:10:25.898 INFO:teuthology.orchestra.run.trial094.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T15:10:25.898 INFO:teuthology.orchestra.run.trial094.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-25T15:10:25.899 INFO:teuthology.orchestra.run.trial094.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-25T15:10:25.901 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T15:10:25.902 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T15:10:25.902 INFO:teuthology.orchestra.run.trial189.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-25T15:10:25.902 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T15:10:25.903 INFO:teuthology.orchestra.run.trial189.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-25T15:10:25.904 INFO:teuthology.orchestra.run.trial090.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 83.1% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-25T15:10:25.907 INFO:teuthology.orchestra.run.trial093.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.8% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-25T15:10:25.909 INFO:teuthology.orchestra.run.trial094.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.6% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-25T15:10:25.912 INFO:teuthology.orchestra.run.trial189.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.8% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-25T15:10:25.914 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-25T15:10:25.941 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-25T15:10:25.942 DEBUG:teuthology.orchestra.run.trial090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-25T15:10:25.970 DEBUG:teuthology.orchestra.run.trial093:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-25T15:10:25.995 DEBUG:teuthology.orchestra.run.trial094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-25T15:10:26.021 DEBUG:teuthology.orchestra.run.trial189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-25T15:10:26.047 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-25T15:10:26.064 DEBUG:teuthology.orchestra.run.trial090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-25T15:10:26.067 DEBUG:teuthology.orchestra.run.trial093:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-25T15:10:26.069 DEBUG:teuthology.orchestra.run.trial094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-25T15:10:26.073 DEBUG:teuthology.orchestra.run.trial189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-25T15:10:26.089 INFO:teuthology.orchestra.run.trial090.stdout:kernel.core_pattern = core 2026-02-25T15:10:26.093 INFO:teuthology.orchestra.run.trial094.stdout:kernel.core_pattern = core 2026-02-25T15:10:26.093 INFO:teuthology.orchestra.run.trial093.stdout:kernel.core_pattern = core 2026-02-25T15:10:26.111 INFO:teuthology.orchestra.run.trial189.stdout:kernel.core_pattern = core 2026-02-25T15:10:26.126 DEBUG:teuthology.orchestra.run.trial090:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-25T15:10:26.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T15:10:26.176 DEBUG:teuthology.orchestra.run.trial093:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-25T15:10:26.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T15:10:26.195 DEBUG:teuthology.orchestra.run.trial094:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-25T15:10:26.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T15:10:26.212 DEBUG:teuthology.orchestra.run.trial189:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-25T15:10:26.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T15:10:26.230 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-25T15:10:26.247 INFO:teuthology.task.internal:Transferring archived files... 2026-02-25T15:10:26.248 DEBUG:teuthology.misc:Transferring archived files from trial090:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209/remote/trial090 2026-02-25T15:10:26.249 DEBUG:teuthology.orchestra.run.trial090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-25T15:10:26.287 DEBUG:teuthology.misc:Transferring archived files from trial093:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209/remote/trial093 2026-02-25T15:10:26.287 DEBUG:teuthology.orchestra.run.trial093:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-25T15:10:26.325 DEBUG:teuthology.misc:Transferring archived files from trial094:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209/remote/trial094 2026-02-25T15:10:26.325 DEBUG:teuthology.orchestra.run.trial094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-25T15:10:26.364 DEBUG:teuthology.misc:Transferring archived files from trial189:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/jayaprakash-2026-02-25_14:08:21-rados-tentacle-distro-default-trial/70209/remote/trial189 2026-02-25T15:10:26.364 DEBUG:teuthology.orchestra.run.trial189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-25T15:10:26.417 INFO:teuthology.task.internal:Removing archive directory... 2026-02-25T15:10:26.418 DEBUG:teuthology.orchestra.run.trial090:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-25T15:10:26.421 DEBUG:teuthology.orchestra.run.trial093:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-25T15:10:26.424 DEBUG:teuthology.orchestra.run.trial094:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-25T15:10:26.426 DEBUG:teuthology.orchestra.run.trial189:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-25T15:10:26.450 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-25T15:10:26.465 INFO:teuthology.task.internal:Not uploading archives. 2026-02-25T15:10:26.465 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-25T15:10:26.479 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-25T15:10:26.479 DEBUG:teuthology.orchestra.run.trial090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-25T15:10:26.480 DEBUG:teuthology.orchestra.run.trial093:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-25T15:10:26.482 DEBUG:teuthology.orchestra.run.trial094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-25T15:10:26.483 DEBUG:teuthology.orchestra.run.trial189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-25T15:10:26.494 INFO:teuthology.orchestra.run.trial090.stdout: 920834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 25 15:10 /home/ubuntu/cephtest 2026-02-25T15:10:26.496 INFO:teuthology.orchestra.run.trial093.stdout: 917525 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 25 15:10 /home/ubuntu/cephtest 2026-02-25T15:10:26.497 INFO:teuthology.orchestra.run.trial094.stdout: 917521 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 25 15:10 /home/ubuntu/cephtest 2026-02-25T15:10:26.505 INFO:teuthology.orchestra.run.trial189.stdout: 917525 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 25 15:10 /home/ubuntu/cephtest 2026-02-25T15:10:26.506 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-25T15:10:26.525 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-25T15:10:26.562 INFO:teuthology.run:Summary data: description: rados/verify/{centos_latest ceph clusters/fixed-4 d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/{bluestore/{alloc$/{btree} base mem$/{normal-2} onode-segment$/{none} write$/{random/{compr$/{no$/{no}} random}}}} rados read-affinity/balance tasks/rados_api_tests validater/valgrind} duration: 3176.6605546474457 flavor: default owner: scheduled_jayaprakash@soko04.front.sepia.ceph.com success: true 2026-02-25T15:10:26.562 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-25T15:10:26.679 INFO:teuthology.run:pass